00010000:02000000:0.1F:1787820973.190223:0:20167:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1F:1787820974.252556:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787820975.276618:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787820976.300380:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:1.1F:1787820995.633555:0:35095:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787820996.653537:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787820997.676336:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787820998.700850:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787820999.724292:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:3.1F:1787820999.741445:0:35095:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787821000.748346:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787821001.772455:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787821002.796302:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:2.1:1787821003.820604:0:6516:0:(ldlm_lockd.c:390:__ldlm_add_waiting_lock()) *** cfs_fail_loc=512, val=0*** 00010000:02000000:3.1:1787821009.923416:0:8759:0:(ldlm_lockd.c:391:__ldlm_add_waiting_lock()) *** cfs_fail_loc=511, val=0*** 00010000:02000000:3.1:1787821009.932327:0:8759:0:(ldlm_lockd.c:391:__ldlm_add_waiting_lock()) *** cfs_fail_loc=511, val=0*** 00010000:02000000:1.1:1787821024.028585:0:20167:0:(ldlm_lockd.c:391:__ldlm_add_waiting_lock()) *** cfs_fail_loc=511, val=0*** 00010000:00010000:1.0:1787826244.677201:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f486d6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.677207:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826244.677210:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f486d6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.677216:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f486d6 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.677297:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe400) 00010000:00010000:1.0:1787826244.677301:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826244.677304:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f486d6 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.677312:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f486d6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.677550:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f486c8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800aee expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.677567:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f486dd lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800aee expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.677575:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e38:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826244.677578:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826244.677580:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826244.677757:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222513:x1874663332466560:12345-192.168.203.56@tcp:101:bash.0 Request processed in 9810us (9997us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826244.677766:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68062 00000100:00100000:1.0:1787826244.680705:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826244.685318:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332466816 00000100:00100000:1.0:1787826244.685328:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332466816 00000100:00100000:1.0:1787826244.685385:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68064 00000100:00100000:1.0:1787826244.685397:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222513:x1874663332466816:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826244.685417:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826244.685437:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f486f2 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800afc expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.685576:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f486f9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.685590:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f486f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.685600:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f486f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826244.685645:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18a:0x0]: returning size 4096 00000004:00200000:1.0:1787826244.685650:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826244.685687:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f486f2 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800afc expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.685704:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f486f9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800afc expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.685712:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:1.0:1787826244.685715:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826244.685719:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826244.685780:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222513:x1874663332466816:12345-192.168.203.56@tcp:101:bash.0 Request processed in 386us (574us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826244.685787:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68064 00000100:00100000:1.0:1787826244.691800:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346035840 00000100:00100000:1.0:1787826244.691804:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346035840 00000100:00100000:1.0:1787826244.691871:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7767 00000100:00100000:1.0:1787826244.691877:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d814a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346035840:12345-0@lo:104: 00010000:00010000:1.0:1787826244.691888:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f486ac lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f486b3 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.691944:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826244.691950:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d814a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346035840:12345-0@lo:104: Request processed in 74us (518us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826244.691957:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7767 00000100:00100000:1.0:1787826244.697212:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826244.727525:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332467840 00000100:00100000:1.0:1787826244.727540:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332467840 00000100:00100000:1.0:1787826244.727622:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68067 00000100:00100000:1.0:1787826244.727632:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222513:x1874663332467840:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826244.727653:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826244.727656:0:11629:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787826244.727684:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f486f9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800afc expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.727706:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:1.0:1787826244.727712:0:11629:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826244.727724:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48738 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800b18 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.727983:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.727997:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.728005:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.728021:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f48746 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.728028:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f48746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.728035:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f48746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826244.728247:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346036096:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787826244.728313:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826244.728470:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826244.732609:0:11629:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957253 gen 2 00000004:00080000:1.0:1787826244.732618:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957253 committed transno 8589957253 00000100:00100000:1.0:1787826244.732652:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346036096:0@lo:1000:mdt00_004.0 00000004:00080000:1.0:1787826244.732938:0:11629:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4c5:0x0], index 1: 0 00010000:00010000:1.0:1787826244.733099:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4874d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733156:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4874d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733171:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4874d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733209:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733216:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733222:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733229:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733234:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826244.733239:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733245:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733257:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:1.0:1787826244.733260:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826244.733263:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733269:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4873f lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826244.733284:0:11629:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f24f69b1 reply state 00000000c82916aa transno 4294985276 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f48746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733296:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f48746 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733303:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x4ec94fd6 (ffff9f7967ac1200) 00010000:00010000:1.0:1787826244.733305:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826244.733307:0:11629:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826244.733550:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48738 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b18 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733568:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4874d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b18 expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.733576:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e39:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:1.0:1787826244.733579:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826244.733582:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826244.733653:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:222513:x1874663332467840:12345-192.168.203.56@tcp:101:bash.0 Request processed in 6022us (11935us total) trans 4294985276 rc 0/0 00000100:00100000:1.0:1787826244.733663:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68067 00000100:00100000:1.0:1787826244.740265:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332467968 00000100:00100000:1.0:1787826244.740274:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332467968 00000100:00100000:1.0:1787826244.740344:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30741 00000100:00100000:1.0:1787826244.740355:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222513:x1874663332467968:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826244.740374:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826244.740405:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000975d3a2a/0xe21309aa81f4875b lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800b1f expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826244.740431:0:24218:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4c5:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787826244.740452:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c5:0x0].0x0 since=8056s v=(0 0) 00010000:00010000:1.0:1787826244.740456:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826244.740466:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000975d3a2a/0xe21309aa81f4875b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b1f expref: 8 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.740482:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000975d3a2a/0xe21309aa81f4875b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b1f expref: 8 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.740494:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826244.740557:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222513:x1874663332467968:12345-192.168.203.56@tcp:101:bash.0 Request processed in 204us (817us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826244.740563:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30741 00000100:00100000:1.0:1787826247.663699:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346036224:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826247.663769:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826247.663840:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346036224 00000100:00100000:1.0:1787826247.663847:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346036224 00000100:00100000:1.0:1787826247.663889:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10924 00000100:00100000:1.0:1787826247.663897:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346036224:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826247.663908:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826247.663925:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f485e8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x187:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f485e1 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.663938:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x187:0x0].0x5a734ce9 (ffff9f7967ac1c00) 00010000:00010000:1.0:1787826247.663956:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826247.663985:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346036224:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 88us (216us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826247.663995:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10924 00000100:00100000:1.0:1787826247.664027:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346036224:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:1.0:1787826247.664054:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3373 committed batchid 3365 00000020:00080000:1.0:1787826247.664160:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3365 00000100:00100000:1.0:1787826247.664229:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826247.664236:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346036352:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826247.664262:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826247.664289:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346036352 00000100:00100000:1.0:1787826247.664292:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346036352 00000100:00100000:1.0:1787826247.664339:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30742 00000100:00100000:1.0:1787826247.664347:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346036352:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1787826247.664368:0:8443:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1223 count 1 00010000:00010000:1.0:1787826247.664389:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48762 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.664407:0:8443:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f483f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800a38 expref: 8 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.664416:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4c7:0x0].0x0 since=9s v=(0 0) 00010000:00010000:1.0:1787826247.664421:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826247.664425:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48762 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.664452:0:8443:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f483f7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800a38 expref: 8 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.664471:0:8443:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f483f7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800a38 expref: 8 pid: 35096 timeout: 8159 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826247.664483:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346036480:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826247.664532:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705840 going to sleep for 11 seconds 00000100:00080000:1.0:1787826247.664543:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787826247.684735:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f483f7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800a38 expref: 8 pid: 35096 timeout: 8159 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826247.684759:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346036480:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826247.684789:0:8443:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48762 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.684803:0:8443:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48762 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.684810:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48762 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.684820:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48762 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.684828:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48762 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826247.684861:0:8443:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826247.685068:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346036352:12345-0@lo:6:osp-syn-0-1.0 Request processed in 20720us (20802us total) trans 4294973826 rc 0/0 00000100:00100000:1.0:1787826247.685077:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30742 00000100:00100000:1.0:1787826247.912316:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332469376 00000100:00100000:1.0:1787826247.912321:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332469376 00000100:00100000:1.0:1787826247.912377:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8720 00000100:00100000:1.0:1787826247.912387:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332469376:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826247.912427:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332469376:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 44us (861us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826247.912434:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8720 00000100:00100000:1.0:1787826248.686671:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346036608:0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.686709:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826248.686750:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346037120:0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.686778:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826248.690517:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346037632:0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.690581:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826248.690619:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346037632 00000100:00100000:1.0:1787826248.690624:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346037632 00000100:00100000:1.0:1787826248.690666:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8723 00000100:00100000:1.0:1787826248.690679:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346037632:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.690965:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346037632:12345-0@lo:400:kworker.0 Request processed in 288us (383us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826248.690972:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8723 00000100:00100000:1.0:1787826248.691004:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346036992:0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.691036:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826248.691070:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346037504:0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.691134:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826248.691159:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346038016:0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.691192:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826248.691246:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346036992 00000100:00100000:1.0:1787826248.691251:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346036992 00000100:00100000:1.0:1787826248.691299:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346037504 00000100:00100000:1.0:1787826248.691303:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346037504 00000100:00100000:1.0:1787826248.694498:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14179 00000100:00100000:1.0:1787826248.694509:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346037504:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.694591:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346037504:12345-0@lo:400:kworker.0 Request processed in 84us (3458us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826248.694599:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14179 00000100:00100000:1.0:1787826248.694619:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14180 00000100:00100000:1.0:1787826248.694628:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346038144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826248.694664:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346038144:12345-0@lo:400:kworker.0 Request processed in 39us (1292us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826248.694671:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14180 00000100:00100000:1.0:1787826248.765654:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826248.779914:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826248.787932:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826248.813910:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826248.926503:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826248.943676:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332470912:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 12181us (14615us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826248.943691:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30743 00000100:00100000:1.0:1787826249.198579:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346038272 00000100:00100000:1.0:1787826249.198587:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346038272 00000100:00100000:1.0:1787826249.198645:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6175 00000100:00100000:1.0:1787826249.198655:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346038272:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826249.198678:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826249.198729:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346038272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 75us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.198736:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6175 00000100:00100000:1.0:1787826249.198779:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346038272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826249.279525:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332471296 00000100:00100000:1.0:1787826249.279536:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332471296 00000100:00100000:1.0:1787826249.279613:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68074 00000100:00100000:1.0:1787826249.279625:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222696:x1874663332471296:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826249.279646:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826249.279667:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f487d2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800b50 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.279733:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.279748:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.279760:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826249.279814:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826249.279818:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826249.279837:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.279861:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f487d2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b50 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.279881:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b50 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.279890:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826249.279893:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826249.279896:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826249.279947:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222696:x1874663332471296:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 325us (647us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.279954:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68074 00000100:00100000:1.0:1787826249.301668:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332471680 00000100:00100000:1.0:1787826249.301678:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332471680 00000100:00100000:1.0:1787826249.301747:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68077 00000100:00100000:1.0:1787826249.301760:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222696:x1874663332471680:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826249.301781:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826249.301804:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4880a lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800b65 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.301878:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.301897:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.301908:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.301925:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.301932:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.301939:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302027:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4881f lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302036:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4881f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302043:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4881f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826249.302100:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e39:0x0]: returning size 6 00000004:00200000:1.0:1787826249.302126:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826249.302148:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4881f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302161:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4881f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302175:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302182:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302188:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302194:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302201:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826249.302205:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302213:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302224:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x0 (ffff9f79462e8400) 00010000:00010000:1.0:1787826249.302228:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826249.302232:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302238:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48811 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302250:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302257:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302263:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302268:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826249.302271:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302277:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302284:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x4ec94fd6 (ffff9f7967ac1c00) 00010000:00010000:1.0:1787826249.302286:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826249.302289:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302294:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f48818 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302323:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4880a lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b65 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302336:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4881f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b65 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.302343:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e39:0x0].0x0 (ffff9f7967ac1200) 00010000:00010000:1.0:1787826249.302345:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826249.302348:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826249.302405:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222696:x1874663332471680:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 648us (863us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.302416:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68077 00000100:00100000:1.0:1787826249.315189:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332471808 00000100:00100000:1.0:1787826249.315199:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332471808 00000100:00100000:1.0:1787826249.315259:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68078 00000100:00100000:1.0:1787826249.315271:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:222696:x1874663332471808:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826249.315293:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826249.315313:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48826 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800b6c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315374:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315389:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315401:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315411:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315419:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315425:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315798:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4883b lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315811:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4883b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315818:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4883b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826249.315859:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db0:0x0]: returning size 6 00000004:00200000:1.0:1787826249.315863:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826249.315880:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4883b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315892:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4883b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315901:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315907:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315913:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315918:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315923:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826249.315927:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315935:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315946:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x0 (ffff9f7843326300) 00010000:00010000:1.0:1787826249.315949:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826249.315953:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 3/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315959:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4882d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315972:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315979:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315985:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315990:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826249.315993:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.315999:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.316006:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x5a734ce9 (ffff9f7843326d00) 00010000:00010000:1.0:1787826249.316008:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826249.316011:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.316016:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f48834 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.316045:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48826 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b6c expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.316058:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4883b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b6c expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.316065:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db0:0x0].0x0 (ffff9f7967ac1300) 00010000:00010000:1.0:1787826249.316067:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826249.316070:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826249.316166:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:222696:x1874663332471808:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 896us (1021us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.316174:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68078 00000100:00100000:1.0:1787826249.321196:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826249.321235:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826249.322325:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038400:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.322395:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038528:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.322426:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.328355:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bcd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826249.331171:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038528:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826249.331215:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18a:0x0] 00000100:00100000:1.0:1787826249.331273:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038656:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.331308:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.331333:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826249.331349:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346038656 00000100:00100000:1.0:1787826249.331353:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346038656 00000100:00100000:1.0:1787826249.331380:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23013 00000100:00100000:1.0:1787826249.331387:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346038656:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.331447:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346038656:12345-0@lo:1000:mdt_io00_001.0 Request processed in 60us (138us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.331454:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23013 00000100:00100000:1.0:1787826249.331480:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038656:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826249.331498:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e38:0x0] 00000100:00100000:1.0:1787826249.331524:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038784:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.331547:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.331561:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bcd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826249.331573:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346038784 00000100:00100000:1.0:1787826249.331576:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346038784 00000100:00100000:1.0:1787826249.331600:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23014 00000100:00100000:1.0:1787826249.331606:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346038784:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.331655:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346038784:12345-0@lo:1000:mdt_io00_001.0 Request processed in 49us (107us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.331661:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23014 00000100:00100000:1.0:1787826249.331685:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038784:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826249.331716:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18a:0x0] 00000100:00100000:1.0:1787826249.331732:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038912:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.331755:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.331768:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826249.331779:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346038912 00000100:00100000:1.0:1787826249.331783:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346038912 00000100:00100000:1.0:1787826249.331802:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23015 00000100:00100000:1.0:1787826249.331808:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346038912:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.331853:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346038912:12345-0@lo:1000:mdt_io00_001.0 Request processed in 45us (97us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.331860:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23015 00000100:00100000:1.0:1787826249.331885:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346038912:0@lo:1000:mdt_io00_001.0 00000004:00020000:1.0:1787826249.331909:0:6541:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:1.0:1787826249.331963:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48842 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.331986:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48842 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.331996:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48842 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826249.332008:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346039040:0@lo:101:mdt_io00_001.0 00000100:00100000:1.0:1787826249.333912:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.333950:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826249.374019:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346039808 00000100:00100000:1.0:1787826249.374040:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346039808 00000100:00100000:1.0:1787826249.374129:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23018 00000100:00100000:1.0:1787826249.374146:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346039808:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.374245:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346039808:12345-0@lo:1000:mdt_io00_001.0 Request processed in 105us (342us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.382546:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23018 00000100:00100000:1.0:1787826249.385844:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:125406:1874663346040192:192.168.203.56@tcp:105: 00010000:00010000:1.0:1787826249.385864:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826249.385905:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332472832:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3209us (4639us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.385917:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21778 00000100:00100000:1.0:1787826249.405471:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332473088 00000100:00100000:1.0:1787826249.405481:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332473088 00010000:00010000:1.0:1787826249.405519:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48849 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b49 expref: 10 pid: 8752 timeout: 8161 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826249.405562:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21779 00000100:00100000:1.0:1787826249.405573:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332473088:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826249.405588:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826249.405595:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48849 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b49 expref: 10 pid: 8752 timeout: 8161 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.405607:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48849 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b49 expref: 10 pid: 8752 timeout: 8161 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.405629:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48849 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db0:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b49 expref: 10 pid: 8752 timeout: 8161 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.405645:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db0:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:1.0:1787826249.405650:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48888 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.405659:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48888 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.405666:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48888 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.405703:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826249.405761:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332473088:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 189us (1296us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.405770:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21779 00000100:00100000:1.0:1787826249.405982:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346040832 00000100:00100000:1.0:1787826249.405987:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346040832 00000100:00100000:1.0:1787826249.406029:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23023 00000100:00100000:1.0:1787826249.406039:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346040832:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826249.406054:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826249.406069:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f48896 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4888f expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.406097:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f48896 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4888f expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.406126:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f48896 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4888f expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.406134:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e39:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:1.0:1787826249.406137:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787826249.406140:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826249.406182:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346040832:12345-0@lo:101:mdt_io00_001.0 Request processed in 146us (287us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.406188:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23023 00000100:00100000:1.0:1787826249.407645:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346040960:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826249.407676:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18a:0x0] 00000100:00100000:1.0:1787826249.407784:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346041088:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.407819:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.407854:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826249.410362:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346041216:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826249.410385:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e38:0x0] 00000100:00100000:1.0:1787826249.410411:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346041344:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.410439:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.410463:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453d1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826249.410677:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346041344:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.410711:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346041472:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.410737:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.410759:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826249.411144:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346041600 00000100:00100000:1.0:1787826249.411149:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346041600 00000100:00100000:1.0:1787826249.411179:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23029 00000100:00100000:1.0:1787826249.411185:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346041600:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826249.411234:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346041600:12345-0@lo:1000:mdt_io00_001.0 Request processed in 49us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.411240:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23029 00000100:00100000:2.0:1787826249.417741:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3652:x1874663346042368:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826249.417757:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826249.417778:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48857 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48842 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826249.417795:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x0 (ffff9f79462e8400) 00010000:00010000:2.0:1787826249.417806:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826249.417868:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346042368:12345-0@lo:103:ldlm_bl.0 Request processed in 128us (1559us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826249.417875:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10926 00000100:00100000:2.0:1787826249.423619:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332473472 00000100:00100000:2.0:1787826249.423629:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332473472 00000100:00100000:2.0:1787826249.423688:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9270 00000100:00100000:2.0:1787826249.423700:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222694:x1874663332473472:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:2.0:1787826249.423907:0:6530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1db0:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1787826249.424049:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].208, 64 off21504 00000004:00080000:2.0:1787826249.424062:0:6530:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8061635955829ns, next at 8062635953317ns, rc = 0 00010000:00010000:2.0:1787826249.424155:0:6530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826249.424171:0:6530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826249.424182:0:6530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826249.424190:0:6530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826249.424197:0:6530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826249.424203:0:6530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826249.424223:0:6530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826249.424312:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:222694:x1874663332473472:12345-192.168.203.56@tcp:35:cat.0 Request processed in 613us (913us total) trans 8589957258 rc 0/0 00000100:00100000:2.0:1787826249.424321:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9270 00010000:00010000:0.0:1787826249.424331:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.424344:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826249.424348:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.424356:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.424813:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db0:0x0].0x0 (ffff9f794b57a100) 00010000:00010000:0.0:1787826249.424819:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826249.424825:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.424831:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f488ab lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.424845:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826249.424847:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826249.424849:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826249.708448:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826249.708477:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346042496:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826249.708549:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826249.708598:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346042496 00000100:00100000:0.0:1787826249.708607:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346042496 00000100:00100000:2.0:1787826249.708651:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346042624:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826249.708656:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23034 00000100:00100000:0.0:1787826249.708667:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346042496:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826249.708683:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00000024:0.0:1787826249.708702:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443463168 avail 1297223680 00000100:00100000:0.0:1787826249.708766:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346042496:12345-0@lo:41:osp-pre-1-0.0 Request processed in 100us (216us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826249.708774:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23034 00000100:00100000:0.0:1787826249.708792:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346042624 00000100:00100000:2.0:1787826249.708792:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346042496:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826249.708797:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346042624 00000100:00100000:0.0:1787826249.708824:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23035 00000100:00100000:0.0:1787826249.708833:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346042624:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787826249.708850:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442844672 avail 1296605184 00000100:00100000:0.0:1787826249.708880:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346042624:12345-0@lo:41:osp-pre-0-1.0 Request processed in 49us (198us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826249.708887:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23035 00000100:00100000:2.0:1787826249.708905:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346042624:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826249.709102:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346042752:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826249.709136:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826249.709147:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346042880:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826249.709157:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826249.709221:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826249.709229:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346043008:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826249.709463:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826249.709500:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346042752 00000100:00100000:1.0:1787826249.709505:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346042752 00000100:00100000:1.0:1787826249.709538:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6176 00000100:00100000:1.0:1787826249.709549:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346042752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826249.709567:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662401536 00000100:00100000:2.0:1787826249.709626:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826249.709773:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346042880:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826249.709811:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973826 gen 2 00000004:00080000:2.0:1787826249.709818:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7850ad4e00, transno 4294973826 00000100:00100000:2.0:1787826249.709834:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346043008:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826249.710005:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346042752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 458us (848us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826249.710011:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346042752:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826249.710017:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6176 00000100:00100000:1.0:1787826249.847827:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346043136:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826249.847881:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826249.847925:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4871c lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.847935:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4871c lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.847943:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4871c lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.847951:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826249.847957:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4871c lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.847966:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4871c lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.847982:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x5a734ce9 (ffff9f7843326d00) 00010000:00010000:1.0:1787826249.847986:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826249.847990:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4871c lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.847996:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4871c lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826249.848028:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346043264 00000100:00100000:1.0:1787826249.848036:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346043264 00000100:00100000:1.0:1787826249.848093:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346043136 00000100:00100000:1.0:1787826249.848097:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346043136 00000100:00100000:0.0:1787826249.848125:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10927 00000100:00100000:0.0:1787826249.848139:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346043264:12345-0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826249.848145:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10928 00000100:00100000:1.0:1787826249.848153:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346043136:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826249.848157:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826249.848162:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826249.848177:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4886c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18a:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4885e expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.848181:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f48896 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4888f expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826249.848190:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x4ec94fd6 (ffff9f794b57ad00) 00010000:00010000:0.0:1787826249.848200:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e39:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:1.0:1787826249.848201:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826249.848213:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826249.848239:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346043136:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 86us (358us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826249.848246:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10928 00000100:00100000:0.0:1787826249.848268:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346043264:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 133us (416us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826249.848276:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10927 00000100:00100000:1.0:1787826249.848298:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346043136:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826249.848315:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346043264:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826249.849305:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346043392:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1787826249.849362:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:0.0:1787826249.849401:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979071 00000004:00080000:0.0:1787826249.849460:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7841944300 with version 1805 00000020:00080000:0.0:1787826249.849469:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979071 cancel update log [0x240000404:0x2:0x0].927: rc = 0 00000020:00080000:0.0:1787826249.849544:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979071 cancel update log [0x200000bd0:0x2:0x0].924: rc = 0 00000020:00080000:0.0:1787826249.849554:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979071 00000100:00100000:0.0:1787826249.849568:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346043392 00000100:00100000:0.0:1787826249.849573:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346043392 00000100:00100000:0.0:1787826249.849610:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30747 00000100:00100000:0.0:1787826249.849617:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346043392:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1787826249.849644:0:8759:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1306 count 1 00010000:00010000:0.0:1787826249.849667:0:8759:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a433ba8e/0xe21309aa81f488b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.849843:0:8759:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ae043674/0xe21309aa81f48881 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b7a expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.849853:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51a:0x0].0x0 since=1s v=(1 0) 00010000:00010000:0.0:1787826249.849857:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826249.849861:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a433ba8e/0xe21309aa81f488b2 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x51a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.849889:0:8759:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ae043674/0xe21309aa81f48881 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b7a expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.849903:0:8759:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ae043674/0xe21309aa81f48881 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b7a expref: 5 pid: 8443 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826249.849915:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346043520:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826249.849976:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc780 going to sleep for 11 seconds 00000004:00080000:0.0:1787826249.849988:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1805 rpc_version 1805 00000020:00100000:0.0:1787826249.850007:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826249.850014:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5a6680 x1874663346043776/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826249.850028:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a6680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346043776:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826249.850056:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826249.850073:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852b9c900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826249.850649:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346043776 00000100:00100000:0.0:1787826249.850657:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346043776 00000100:00100000:0.0:1787826249.850769:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23036 00000100:00100000:0.0:1787826249.850779:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346043776:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826249.850923:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346043776:12345-0@lo:1000:osp_up1-0.0 Request processed in 146us (865us total) trans 4294985279 rc 0/0 00000100:00100000:0.0:1787826249.850930:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23036 00000100:00100000:0.0:1787826249.850979:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a6680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346043776:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826249.855086:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346043520:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826249.855135:0:8759:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a433ba8e/0xe21309aa81f488b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.859788:0:8759:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a433ba8e/0xe21309aa81f488b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.859799:0:8759:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a433ba8e/0xe21309aa81f488b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.859811:0:8759:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a433ba8e/0xe21309aa81f488b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.859820:0:8759:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a433ba8e/0xe21309aa81f488b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826249.859870:0:8759:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826249.860343:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346043392:12345-0@lo:6:osp-syn-0-0.0 Request processed in 10559us (10814us total) trans 4294973828 rc 0/0 00000100:00100000:0.0:1787826249.860354:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30747 00000004:00080000:0.0:1787826249.860679:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f78435b2680/2, rc 0, transno 4294973828 00000100:00100000:0.0:1787826249.860686:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346043392:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787826250.492206:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332473728 00000100:00100000:1.0:1787826250.492216:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332473728 00000100:00100000:1.0:1787826250.492272:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68080 00000100:00100000:1.0:1787826250.492282:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:222732:x1874663332473728:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826250.492301:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826250.492334:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f488b9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800b88 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.492401:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f488c0 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.492419:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f488c0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.492430:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f488c0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826250.492521:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e38:0x0]: returning size 4096 00000004:00200000:1.0:1787826250.492525:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787826250.492543:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f488c0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.492574:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f488b9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b88 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.492594:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f488c0 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b88 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.492602:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e38:0x0].0x0 (ffff9f7843326200) 00010000:00010000:1.0:1787826250.492605:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826250.492610:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826250.492670:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:222732:x1874663332473728:12345-192.168.203.56@tcp:101:rm.0 Request processed in 388us (675us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.492677:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68080 00000100:00100000:1.0:1787826250.502359:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68081 00000100:00100000:1.0:1787826250.502490:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:222732:x1874663332473856:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826250.502510:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826250.502534:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f488c7 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800b8f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.502596:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f488ce lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.502610:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f488ce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.502621:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f488ce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826250.502666:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18a:0x0]: returning size 4096 00000004:00200000:1.0:1787826250.502671:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826250.502703:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f488c7 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b8f expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.502720:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f488ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b8f expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.502728:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:1.0:1787826250.502730:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826250.502733:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826250.502814:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:222732:x1874663332473856:12345-192.168.203.56@tcp:101:rm.0 Request processed in 329us (779us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.502821:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68081 00000100:00100000:0.0:1787826250.507431:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332473984 00000100:00100000:0.0:1787826250.507440:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332473984 00000100:00100000:0.0:1787826250.507507:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68082 00000100:00100000:0.0:1787826250.507518:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222732:x1874663332473984:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826250.507542:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826250.507566:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f488d5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800b96 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.507632:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x188:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.507645:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.507656:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826250.507704:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x188:0x0]: returning size 4096 00000004:00200000:0.0:1787826250.507709:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826250.507743:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f488d5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.507766:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.507774:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x0 (ffff9f7845784800) 00010000:00010000:0.0:1787826250.507777:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826250.507781:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826250.507837:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222732:x1874663332473984:12345-192.168.203.56@tcp:101:rm.0 Request processed in 321us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.507845:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68082 00000100:00100000:0.0:1787826250.514496:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826250.523402:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.544691:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332474496 00000100:00100000:1.0:1787826250.544702:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332474496 00000100:00100000:1.0:1787826250.544765:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9272 00000100:00100000:1.0:1787826250.544775:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943005c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222732:x1874663332474496:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826250.549554:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943005c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222732:x1874663332474496:12345-192.168.203.56@tcp:37:rm.0 Request processed in 4770us (5056us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.549569:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9272 00000100:00100000:1.0:1787826250.560688:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000004:00080000:1.0:1787826250.568776:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6744 rpc_version 6744 00000020:00100000:1.0:1787826250.568825:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826250.568838:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7854625880 x1874663346044032/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826250.568865:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854625880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346044032:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826250.571092:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826250.571168:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346044032 00000100:00100000:2.0:1787826250.571179:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346044032 00000100:00100000:1.0:1787826250.571181:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826250.571247:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23037 00000100:00100000:2.0:1787826250.571257:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346044032:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826250.571538:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346044032:12345-0@lo:1000:osp_up0-1.0 Request processed in 281us (451us total) trans 8589957259 rc 0/0 00000100:00100000:2.0:1787826250.571546:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23037 00000100:00100000:2.0:1787826250.571611:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854625880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346044032:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826250.571723:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222732:x1874663332474752:12345-192.168.203.56@tcp:35:rm.0 Request processed in 10879us (11044us total) trans 4294985281 rc 0/0 00000100:00100000:2.0:1787826250.571732:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9273 00000100:00100000:0.0:1787826250.584603:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826250.584738:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332475008 00000100:00100000:2.0:1787826250.584749:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332475008 00000100:00100000:2.0:1787826250.584810:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68084 00000100:00100000:2.0:1787826250.584821:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222732:x1874663332475008:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826250.584845:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826250.584908:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f488f1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ba4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.585170:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f488f8 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e38:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.585184:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f488f8 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e38:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.585195:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f488f8 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e38:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826250.585205:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e38:0x0]: returning size 4096 00010000:00010000:2.0:1787826250.585287:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f488f1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ba4 expref: 13 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.585316:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f488f8 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e38:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ba4 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.585324:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e38:0x0].0x0 (ffff9f7843326200) 00010000:00010000:2.0:1787826250.585327:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1787826250.585330:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826250.585380:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:222732:x1874663332475008:12345-192.168.203.56@tcp:101:rm.0 Request processed in 562us (784us total) trans 4294985282 rc 0/0 00000100:00100000:2.0:1787826250.585387:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68084 00000100:00100000:0.0:1787826250.591723:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.591814:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332475136 00000100:00100000:1.0:1787826250.591824:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332475136 00000100:00100000:1.0:1787826250.591884:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68085 00000100:00100000:1.0:1787826250.591896:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222732:x1874663332475136:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:1.0:1787826250.591957:0:24703:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:1.0:1787826250.592018:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222732:x1874663332475136:12345-192.168.203.56@tcp:41:rm.0 Request processed in 125us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.592026:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68085 00000100:00100000:1.0:1787826250.596300:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826250.596430:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332475264 00000100:00100000:0.0:1787826250.596440:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332475264 00000100:00100000:0.0:1787826250.596501:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68086 00000100:00100000:0.0:1787826250.596512:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222732:x1874663332475264:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826250.596532:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826250.596553:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f48906 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800bab expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596620:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596639:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596650:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596666:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596673:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596680:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596740:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4891b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596749:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4891b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596757:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4891b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826250.596778:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4891b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596792:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596799:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596811:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596817:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596823:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826250.596828:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596836:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596847:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x0 (ffff9f7845784800) 00010000:00010000:0.0:1787826250.596850:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826250.596854:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 3/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596861:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4890d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596873:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596879:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596885:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596890:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826250.596893:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596899:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596906:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x5a734ce9 (ffff9f7845784600) 00010000:00010000:0.0:1787826250.596908:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826250.596911:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596917:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f48914 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596949:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f48906 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bab expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596963:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4891b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bab expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.596970:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e39:0x0].0x0 (ffff9f7845784d00) 00010000:00010000:0.0:1787826250.596972:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826250.596975:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826250.597037:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:222732:x1874663332475264:12345-192.168.203.56@tcp:101:rm.0 Request processed in 527us (747us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.597045:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68086 00000100:00100000:0.0:1787826250.617403:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826250.617443:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332475520 00000100:00100000:0.0:1787826250.617452:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332475520 00000100:00100000:0.0:1787826250.617511:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10929 00000100:00100000:0.0:1787826250.617522:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b470050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332475520:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826250.617537:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826250.617564:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4891b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bab expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.617579:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e39:0x0].0x0 (ffff9f7845784d00) 00010000:00010000:0.0:1787826250.617589:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826250.617647:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b470050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332475520:12345-192.168.203.56@tcp:103:rm.0 Request processed in 127us (253us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.617655:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10929 00000100:00100000:1.0:1787826250.618925:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346044160 00000100:00100000:1.0:1787826250.618931:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346044160 00000100:00100000:1.0:1787826250.618967:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23038 00000100:00100000:1.0:1787826250.618975:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35093:x1874663346044160:12345-0@lo:101:mdt00_010.0 00010000:00010000:1.0:1787826250.618989:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826250.619002:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f48937 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48930 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.619025:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.619035:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f48937 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48930 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.619059:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.619085:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 9 pid: 35095 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826250.619097:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346044288:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826250.619317:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826250.637995:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826250.638315:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332475904 00000100:00100000:0.0:1787826250.638325:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332475904 00010000:00010000:0.0:1787826250.638365:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 10 pid: 35095 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.638400:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21781 00000100:00100000:0.0:1787826250.638409:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332475904:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826250.638423:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826250.638431:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 10 pid: 35095 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638444:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 10 pid: 35095 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638468:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f488dc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x188:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b96 expref: 10 pid: 35095 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638489:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x0 (ffff9f7845784800) 00010000:00010000:0.0:1787826250.638497:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f48937 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48930 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638509:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f48937 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48930 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638519:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f48937 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48930 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638551:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f48937 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48930 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.638567:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346044416:0@lo:105: 00000100:00100000:0.0:1787826250.638615:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826250.638649:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ea80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826250.638675:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826250.638685:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346044416 00000100:00100000:0.0:1787826250.638688:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346044416 00000100:00100000:0.0:1787826250.638714:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7769 00000100:00100000:0.0:1787826250.638721:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843361c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346044416:12345-0@lo:105: 00010000:00010000:0.0:1787826250.638731:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638742:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638757:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638766:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638774:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638784:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.638802:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69400) 00000100:00100000:0.0:1787826250.638834:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843361c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346044416:12345-0@lo:105: Request processed in 113us (219us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.638842:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7769 00000100:00100000:0.0:1787826250.638912:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346044416:0@lo:105: 00010000:00010000:0.0:1787826250.638927:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826250.638961:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332475904:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 553us (975us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.638969:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21781 00000004:00080000:2.0:1787826250.647707:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6745 rpc_version 6745 00000020:00100000:2.0:1787826250.647750:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826250.647761:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f785124f100 x1874663346045312/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826250.647783:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346045312:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826250.647843:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826250.647873:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826250.647894:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346045312 00000100:00100000:2.0:1787826250.647900:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346045312 00000100:00100000:2.0:1787826250.647941:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23044 00000100:00100000:2.0:1787826250.647951:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346045312:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826250.648276:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346045312:12345-0@lo:1000:osp_up0-1.0 Request processed in 325us (432us total) trans 8589957260 rc 0/0 00000100:00100000:2.0:1787826250.648283:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23044 00000100:00100000:2.0:1787826250.648339:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346045312:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826250.648384:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48961 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e39:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648399:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48961 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648411:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48961 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648420:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48961 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648427:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48961 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648435:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48961 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648456:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787826250.648476:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4895a lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648482:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4895a lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648489:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4895a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648495:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826250.648500:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4895a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648508:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4895a lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648521:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e39:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:2.0:1787826250.648524:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826250.648529:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4895a lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648535:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4895a lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648548:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4894c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48953 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648557:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4894c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48953 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648564:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4894c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48953 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.648580:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1787826250.649453:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346045440:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826250.649505:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826250.649557:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346045440 00000100:00100000:0.0:1787826250.649563:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346045440 00000100:00100000:0.0:1787826250.649694:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10930 00000100:00100000:0.0:1787826250.649704:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346045440:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826250.649716:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826250.649734:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f48953 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e39:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4894c expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.649747:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e39:0x0].0x0 (ffff9f7961d81800) 00010000:00010000:0.0:1787826250.649755:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826250.649787:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346045440:12345-0@lo:103:ldlm_bl.0 Request processed in 85us (282us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.649794:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10930 00000100:00100000:0.0:1787826250.649845:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346045440:0@lo:103:ldlm_bl.0 00000004:00080000:2.0:1787826250.650462:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x188:0x0] 00010000:00010000:2.0:1787826250.650475:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650484:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650492:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650511:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826250.650518:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x188:0x0] 00010000:00010000:2.0:1787826250.650524:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4893e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48945 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650531:0:35093:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4893e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48945 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826250.650680:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:222732:x1874663332475776:12345-192.168.203.56@tcp:36:rm.0 Request processed in 32164us (32415us total) trans 4294985283 rc 0/0 00000100:00100000:2.0:1787826250.650689:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68088 00010000:00010000:2.0:1787826250.650721:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826250.650728:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650736:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826250.650742:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826250.650751:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650779:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650785:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826250.650805:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650812:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f48930 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48937 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.650829:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826250.650830:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826250.650832:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826250.650838:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826250.651728:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346045568:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826250.651775:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826250.651820:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346045568 00000100:00100000:2.0:1787826250.651826:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346045568 00000100:00100000:2.0:1787826250.651866:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10931 00000100:00100000:2.0:1787826250.651874:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346045568:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826250.651886:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826250.651904:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f48937 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x188:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48930 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.651919:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x0 (ffff9f7845784800) 00010000:00010000:2.0:1787826250.651928:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826250.651956:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346045568:12345-0@lo:103:ldlm_bl.0 Request processed in 83us (181us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826250.651963:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10931 00000100:00100000:2.0:1787826250.651996:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346045568:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826250.669310:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.669359:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332476544 00000100:00100000:1.0:1787826250.669772:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332476544 00000100:00100000:1.0:1787826250.669841:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68089 00000100:00100000:0.0:1787826250.671980:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:222732:x1874663332476544:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826250.672044:0:24702:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826250.672081:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f488f8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e38:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ba4 expref: 14 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.672130:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f488c0 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b88 expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.672149:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4879a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b34 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.672159:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e38:0x0].0x0 (ffff9f7843326200) 00010000:00010000:0.0:1787826250.672165:0:24702:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826250.672208:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.672217:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.672227:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.672250:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346045696:0@lo:101:mdt00_006.0 00000100:00100000:0.0:1787826250.672313:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826250.672344:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ea80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826250.677182:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346045696:0@lo:101:mdt00_006.0 00010000:00010000:0.0:1787826250.677206:0:24702:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.677218:0:24702:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826250.678188:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.678218:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826250.678288:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332476672 00000100:00100000:0.0:1787826250.678297:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332476672 00010000:00010000:0.0:1787826250.678332:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b50 expref: 8 pid: 6528 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.678366:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332476800 00000100:00100000:0.0:1787826250.678370:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332476800 00010000:00010000:0.0:1787826250.678381:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f48770 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b26 expref: 8 pid: 35095 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.678408:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21782 00000100:00100000:0.0:1787826250.678418:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332476672:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826250.678431:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826250.678438:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b50 expref: 8 pid: 6528 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.678452:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b50 expref: 8 pid: 6528 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.678475:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f487d9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b50 expref: 8 pid: 6528 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.678491:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826250.678497:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4896f lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48968 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.678508:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826250.678552:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332476672:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 134us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.678565:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21782 00000100:00100000:0.0:1787826250.678902:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346046080 00000100:00100000:0.0:1787826250.678907:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346046080 00000100:00100000:0.0:1787826250.678940:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7770 00000100:00100000:0.0:1787826250.678947:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346046080:12345-0@lo:105: 00010000:00010000:0.0:1787826250.678958:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.678969:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.678984:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.678992:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.679000:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.679009:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.679035:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43000) 00000100:00100000:0.0:1787826250.679068:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346046080:12345-0@lo:105: Request processed in 121us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.679088:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7770 00000100:00100000:0.0:1787826250.681457:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346046208 00000100:00100000:0.0:1787826250.681468:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346046208 00000100:00100000:0.0:1787826250.681521:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23046 00000100:00100000:0.0:1787826250.681531:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:24702:x1874663346046208:12345-0@lo:101:mdt00_006.0 00010000:00010000:0.0:1787826250.681550:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826250.681573:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4897d lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48976 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.682182:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4897d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48976 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.682205:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4897d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48976 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.682215:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845784d00) 00010000:00010000:0.0:1787826250.682218:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826250.682221:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826250.682290:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346046208:12345-0@lo:101:mdt00_006.0 Request processed in 761us (2930us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.682298:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23046 00000100:00100000:0.0:1787826250.684178:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346046336 00000100:00100000:0.0:1787826250.684186:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346046336 00000100:00100000:0.0:1787826250.684232:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23047 00000100:00100000:0.0:1787826250.684241:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346046336:12345-0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826250.685467:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346046336:12345-0@lo:1000:mdt00_006.0 Request processed in 1220us (1530us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.685479:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23047 00000100:00100000:1.0:1787826250.687268:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346046336:0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826250.687363:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346046464:0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826250.687421:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826250.687448:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826250.689505:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346046464:0@lo:1000:mdt00_006.0 00010000:00010000:1.0:1787826250.689598:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.689609:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.689619:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826250.689637:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346046592:0@lo:101:mdt00_006.0 00000100:00100000:1.0:1787826250.689689:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826250.689719:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826250.700572:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.701439:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346046592:0@lo:101:mdt00_006.0 00010000:00010000:1.0:1787826250.701461:0:24702:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.701472:0:24702:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.703220:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332476928 00000100:00100000:0.0:1787826250.703233:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332476928 00010000:00010000:0.0:1787826250.703275:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f487f5 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b57 expref: 7 pid: 17813 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.703315:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21784 00000100:00100000:0.0:1787826250.703325:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b472850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332476928:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826250.704197:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826250.704210:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f487f5 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b57 expref: 7 pid: 17813 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.704227:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f487f5 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b57 expref: 7 pid: 17813 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.704250:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f487f5 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b57 expref: 7 pid: 17813 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.704266:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e38:0x0].0x0 (ffff9f7843326000) 00010000:00010000:0.0:1787826250.704274:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4898b lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48984 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.704285:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4898b lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48984 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.704295:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4898b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48984 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.704335:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4898b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48984 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.704362:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346046848:0@lo:105: 00000100:00100000:0.0:1787826250.704423:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826250.704510:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ea80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826250.711460:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346046848 00000100:00100000:1.0:1787826250.711476:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346046848 00000100:00100000:1.0:1787826250.711550:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7771 00000100:00100000:1.0:1787826250.711562:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346046848:12345-0@lo:105: 00010000:00010000:1.0:1787826250.711579:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.711726:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.711738:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.711748:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.711757:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.711768:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.711803:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecee00) 00000100:00100000:1.0:1787826250.711884:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346046848:12345-0@lo:105: Request processed in 324us (7460us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.711892:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7771 00000100:00100000:1.0:1787826250.711941:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346046848:0@lo:105: 00010000:00010000:1.0:1787826250.711961:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826250.712008:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b472850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332476928:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 8681us (11442us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.712017:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21784 00000100:00100000:0.0:1787826250.741414:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.741500:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332477056 00000100:00100000:1.0:1787826250.741510:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332477056 00010000:00010000:1.0:1787826250.741552:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f488ce lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b8f expref: 12 pid: 17813 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826250.741631:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21785 00000100:00100000:1.0:1787826250.741642:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332477056:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826250.741655:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826250.741662:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f488ce lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b8f expref: 12 pid: 17813 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.741677:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f488ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b8f expref: 12 pid: 17813 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.741700:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f488ce lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b8f expref: 12 pid: 17813 timeout: 8162 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.741717:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:1.0:1787826250.741722:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.741732:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.741740:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.741766:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826250.741813:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332477056:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 173us (409us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.741821:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21785 00000004:00080000:0.0:1787826250.752000:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6746 rpc_version 6746 00000020:00100000:0.0:1787826250.752031:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826250.752042:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5a5c00 x1874663346047488/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826250.752065:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346047488:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826250.752162:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826250.752190:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e900 going to sleep for 11 seconds 00000004:00200000:0.0:1787826250.753018:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e38:0x0]: returning size 4096 00010000:00010000:0.0:1787826250.753040:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.753051:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.753059:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.753065:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826250.753080:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.753090:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.753119:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18a:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:0.0:1787826250.754182:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826250.754195:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754204:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f48999 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754232:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754247:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754254:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754280:0:24702:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826250.754290:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x188:0x0] 00010000:00010000:0.0:1787826250.754298:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f48992 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754304:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f48992 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754309:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f48992 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754314:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826250.754318:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f48992 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754324:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f48992 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754336:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e38:0x0].0x0 (ffff9f7843326200) 00010000:00010000:0.0:1787826250.754339:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826250.754342:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f48992 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754348:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f48992 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754359:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754366:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754373:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754438:0:24702:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787826250.754450:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826250.754455:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754462:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.754468:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757214:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787826250.757230:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757243:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826250.757249:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826250.757259:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757302:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757308:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826250.757333:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757340:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f489a0 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f489a7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757364:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826250.757368:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757374:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826250.757377:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826250.757384:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757398:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757404:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826250.757418:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757425:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f48984 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4898b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757436:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826250.757440:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757446:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826250.757450:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826250.757456:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757470:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757475:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826250.757480:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757491:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826250.757493:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826250.757494:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826250.757513:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346047744:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826250.757574:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826250.757610:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826250.757616:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346047872:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826250.757633:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826250.757663:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346047744 00000100:00100000:1.0:1787826250.757669:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346047744 00000100:00100000:1.0:1787826250.757708:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346047872 00000100:00100000:1.0:1787826250.757710:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346047872 00000100:00100000:1.0:1787826250.757728:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10933 00000100:00100000:1.0:1787826250.757736:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+16:3652:x1874663346047744:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826250.757749:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826250.757766:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4898b lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e38:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48984 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757779:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e38:0x0].0x0 (ffff9f7843326000) 00010000:00010000:1.0:1787826250.757787:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826250.757815:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346047744:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.757822:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10933 00000100:00100000:1.0:1787826250.757838:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10934 00000100:00100000:1.0:1787826250.757844:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+14:3652:x1874663346047872:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826250.757850:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826250.757860:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4896f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48968 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.757869:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826250.757872:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826250.757888:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346047872:12345-0@lo:103:ldlm_bl.0 Request processed in 45us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.757894:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10934 00000100:00100000:1.0:1787826250.757923:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346047872:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826250.757931:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346047744:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826250.758031:0:24702:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1787826250.758039:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48968 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4896f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826250.758055:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826250.758062:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.758086:0:24702:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.758167:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:222732:x1874663332476544:12345-192.168.203.56@tcp:36:rm.0 Request processed in 86930us (88864us total) trans 4294985285 rc 0/0 00000100:00100000:0.0:1787826250.758175:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68089 00010000:00010000:1.0:1787826250.758357:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826250.758677:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346047616:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826250.758714:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826250.759252:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346047616 00000100:00100000:1.0:1787826250.759258:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346047616 00000100:00100000:1.0:1787826250.759301:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10935 00000100:00100000:1.0:1787826250.759309:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346047616:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826250.759320:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826250.759344:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f489a7 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x188:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f489a0 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.759359:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:1.0:1787826250.759370:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826250.759412:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346047616:12345-0@lo:103:ldlm_bl.0 Request processed in 103us (697us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.759418:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10935 00000100:00100000:2.0:1787826250.759685:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346047616:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826250.877631:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.877872:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332477184 00000100:00100000:1.0:1787826250.877881:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332477184 00000100:00100000:1.0:1787826250.877941:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68090 00000100:00100000:1.0:1787826250.877951:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222740:x1874663332477184:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:1.0:1787826250.878001:0:11629:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:1.0:1787826250.878080:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222740:x1874663332477184:12345-192.168.203.56@tcp:41:stat.0 Request processed in 129us (457us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.878088:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68090 00010000:00010000:0.0:1787826250.932763:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826250.951230:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.951331:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332477312 00000100:00100000:1.0:1787826250.951351:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332477312 00000100:00100000:1.0:1787826250.951426:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68091 00000100:00100000:1.0:1787826250.951438:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222744:x1874663332477312:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826250.951457:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826250.951478:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f489ae lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800bb9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.951538:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.951552:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.951563:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826250.951618:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826250.951622:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826250.951640:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.951663:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f489ae lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.951688:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.951698:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826250.951701:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826250.951705:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826250.951769:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222744:x1874663332477312:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 334us (549us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.951778:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68091 00000100:00100000:0.0:1787826250.963293:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826250.970251:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332477440 00000100:00100000:2.0:1787826250.970267:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332477440 00000100:00100000:0.0:1787826250.975437:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346048000 00000100:00100000:0.0:1787826250.975451:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346048000 00000100:00100000:0.0:1787826250.975571:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7772 00000100:00100000:0.0:1787826250.975583:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346048000:12345-0@lo:104: 00010000:00010000:0.0:1787826250.975610:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.975713:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826250.975720:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346048000:12345-0@lo:104: Request processed in 140us (2506us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826250.975727:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7772 00000100:00100000:2.0:1787826250.975803:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346048000:0@lo:104: 00010000:00010000:2.0:1787826250.975827:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.975888:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826250.975895:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.975903:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826250.975908:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826250.975917:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.975952:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.975959:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826250.975985:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.975999:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f48976 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4897d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.976021:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826250.976023:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826250.976025:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826250.976032:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826250.976044:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346048128:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826250.976092:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826250.976157:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346048128 00000100:00100000:1.0:1787826250.976162:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346048128 00010000:00010000:1.0:1787826250.976183:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4897d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f48976 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826250.976206:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21786 00000100:00100000:1.0:1787826250.976213:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346048128:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826250.976224:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787826250.978520:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957261 is committed 00010000:00010000:2.0:1787826250.978943:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.978952:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979013:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f489c3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979022:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f489c3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979029:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f489c3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979035:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f489c3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979041:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826250.979046:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f489c3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979053:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f489c3 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979063:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826250.979066:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826250.979081:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f489c3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979087:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f489c3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979097:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979137:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979144:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979149:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826250.979152:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979158:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979166:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845784d00) 00010000:00010000:2.0:1787826250.979168:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826250.979171:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979176:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f489ca lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979220:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f489bc lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc0 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979235:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826250.979238:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826250.979242:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f489bc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc0 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.979251:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:2.0:1787826250.979300:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222744:x1874663332477440:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 7242us (16016us total) trans 0 rc 301/301 00000100:00100000:2.0:1787826250.979308:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68092 00000100:00100000:0.0:1787826250.981279:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346048128:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826250.984657:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826250.989248:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332477824 00000100:00100000:0.0:1787826250.989258:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332477824 00000100:00100000:0.0:1787826250.989318:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68094 00000100:00100000:0.0:1787826250.989329:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222744:x1874663332477824:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:0.0:1787826250.989469:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346048256:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826250.989522:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826250.989555:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826250.989830:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346048256 00000100:00100000:2.0:1787826250.989835:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346048256 00000100:00100000:2.0:1787826250.989873:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23053 00000100:00100000:2.0:1787826250.989882:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35094:x1874663346048256:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826250.989959:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35094:x1874663346048256:12345-0@lo:1000:mdt00_011.0 Request processed in 78us (435us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826250.989966:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23053 00000100:00100000:0.0:1787826250.990544:0:35094:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957261 gen 2 00000004:00080000:0.0:1787826250.990553:0:35094:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957259 committed transno 8589957261 00000004:00080000:0.0:1787826250.990574:0:35094:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957260 committed transno 8589957261 00000004:00080000:0.0:1787826250.990586:0:35094:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957261 committed transno 8589957261 00000100:00100000:0.0:1787826250.990617:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346048256:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826250.990669:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346048384:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826250.990703:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826250.990765:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826250.991702:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346048384:0@lo:1000:mdt00_011.0 00010000:00010000:2.0:1787826250.991761:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.991772:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826250.991782:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826250.991797:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346048512:0@lo:101:mdt00_011.0 00000100:00100000:2.0:1787826250.991828:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826250.992889:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826250.992931:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346048512 00000100:00100000:0.0:1787826250.992937:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346048512 00000100:00100000:0.0:1787826250.992979:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23055 00000100:00100000:0.0:1787826250.992999:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35094:x1874663346048512:12345-0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826250.993011:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826250.993028:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f489e6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f489df expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.993054:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.993064:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f489d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc7 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.993085:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f489e6 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f489df expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.993127:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f489d8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc7 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.993139:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f489d8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc7 expref: 7 pid: 35098 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.993151:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346048640:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826250.993225:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826250.993235:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 6 pid: 6529 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826250.993244:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346048768:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826250.993266:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826250.999151:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826250.999192:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332477952 00000100:00100000:1.0:1787826250.999202:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332477952 00010000:00010000:1.0:1787826250.999242:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f489d8 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc7 expref: 8 pid: 35098 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826250.999294:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21787 00000100:00100000:1.0:1787826250.999303:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332477952:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826250.999316:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826250.999322:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f489d8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc7 expref: 8 pid: 35098 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.999642:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f489d8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc7 expref: 8 pid: 35098 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.999664:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f489d8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bc7 expref: 8 pid: 35098 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.999677:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826250.999682:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f489e6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f489df expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826250.999689:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826250.999742:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332477952:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 440us (601us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826250.999749:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21787 00000100:00100000:0.0:1787826251.004382:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826251.004471:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332478080 00000100:00100000:2.0:1787826251.004480:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332478080 00010000:00010000:2.0:1787826251.004572:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 7 pid: 6529 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826251.004622:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21788 00000100:00100000:2.0:1787826251.004632:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7961cdd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332478080:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826251.004647:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826251.004654:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 7 pid: 6529 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.004666:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 7 pid: 6529 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.004687:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f489b5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bb9 expref: 7 pid: 6529 timeout: 8163 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.004704:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826251.004710:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f489e6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f489df expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.004721:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f489e6 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f489df expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.004730:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f489e6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f489df expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.004765:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f489e6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f489df expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826251.004780:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346048896:0@lo:105: 00000100:00100000:2.0:1787826251.004827:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826251.004872:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826251.005170:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346048896 00000100:00100000:2.0:1787826251.005175:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346048896 00000100:00100000:2.0:1787826251.005205:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7773 00000100:00100000:2.0:1787826251.005212:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346048896:12345-0@lo:105: 00010000:00010000:2.0:1787826251.005222:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.005231:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.005239:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.005246:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.005252:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.005259:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.005281:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490efc00) 00000100:00100000:2.0:1787826251.005320:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346048896:12345-0@lo:105: Request processed in 108us (492us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.005327:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7773 00000100:00100000:2.0:1787826251.005984:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346048896:0@lo:105: 00010000:00010000:2.0:1787826251.005995:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826251.006042:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7961cdd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332478080:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1411us (1667us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.006050:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21788 00000100:00100000:0.0:1787826251.007093:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346049024 00000100:00100000:0.0:1787826251.007132:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346049024 00000100:00100000:0.0:1787826251.007180:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23056 00000100:00100000:0.0:1787826251.007190:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35094:x1874663346049024:12345-0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826251.007208:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826251.007229:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f489f4 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f489ed expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.007253:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f489f4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f489ed expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.007265:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f489f4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f489ed expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.007274:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845784600) 00010000:00010000:0.0:1787826251.007277:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826251.007280:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826251.007348:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049024:12345-0@lo:101:mdt00_011.0 Request processed in 159us (1470us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826251.007356:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23056 00000100:00100000:2.0:1787826251.008276:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346049024:0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826251.008299:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.008311:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826251.008331:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826251.008708:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346049152:0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826251.008757:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826251.008817:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826251.009141:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346049152:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826251.009185:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346049280:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826251.009212:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826251.009242:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826251.009644:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346049280:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826251.009732:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346049408:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826251.009757:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826251.011146:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346049408 00000100:00100000:0.0:1787826251.011148:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826251.011155:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346049408 00000100:00100000:2.0:1787826251.011202:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23059 00000100:00100000:2.0:1787826251.011211:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049408:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826251.012179:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049408:12345-0@lo:1000:mdt00_011.0 Request processed in 965us (2417us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826251.012191:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23059 00000100:00100000:0.0:1787826251.012710:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346049408:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826251.012802:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346049536:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826251.012843:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826251.012874:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826251.013132:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346049536 00000100:00100000:2.0:1787826251.013140:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346049536 00000100:00100000:2.0:1787826251.013179:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23060 00000100:00100000:2.0:1787826251.013187:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049536:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826251.013272:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049536:12345-0@lo:1000:mdt00_011.0 Request processed in 86us (428us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.013279:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23060 00000100:00100000:0.0:1787826251.013627:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346049664 00000100:00100000:0.0:1787826251.013632:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346049664 00000100:00100000:0.0:1787826251.013664:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23061 00000100:00100000:0.0:1787826251.013671:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049664:12345-0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826251.013739:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049664:12345-0@lo:1000:mdt00_011.0 Request processed in 68us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826251.013746:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23061 00000100:00100000:2.0:1787826251.014287:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346049792 00000100:00100000:2.0:1787826251.014292:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346049792 00000100:00100000:2.0:1787826251.014411:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23062 00000100:00100000:2.0:1787826251.014419:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b752d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049792:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826251.014550:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b752d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346049792:12345-0@lo:1000:mdt00_011.0 Request processed in 131us (363us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.014557:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23062 00000004:00080000:0.0:1787826251.016500:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6747 rpc_version 6747 00000020:00100000:0.0:1787826251.016540:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826251.016550:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5a5c00 x1874663346050048/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826251.016571:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346050048:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826251.016649:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826251.016682:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826251.017740:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346050048:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826251.019050:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346050176 00000100:00100000:0.0:1787826251.019057:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346050176 00000100:00100000:0.0:1787826251.019231:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23064 00000100:00100000:0.0:1787826251.019246:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346050176:12345-0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826251.019267:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826251.019286:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f48a10 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48a09 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.019313:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f48a10 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48a09 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.019333:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f48a10 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48a09 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.019361:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x0 (ffff9f7845784b00) 00010000:00010000:0.0:1787826251.019365:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826251.019369:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826251.019427:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346050176:12345-0@lo:101:mdt00_011.0 Request processed in 186us (590us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826251.019433:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23064 00010000:00010000:1.0:1787826251.019704:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826251.019713:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.019724:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efc00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826251.019729:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826251.019738:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.019767:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.019774:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826251.019801:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.019811:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f489df lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f489e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.019824:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826251.019826:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826251.019828:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826251.019837:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826251.019849:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346050304:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826251.019885:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826251.019936:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346050304 00000100:00100000:1.0:1787826251.019942:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346050304 00000100:00100000:1.0:1787826251.020035:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10936 00000100:00100000:1.0:1787826251.020044:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346050304:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826251.020055:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826251.020082:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f489e6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f489df expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.020093:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826251.020097:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826251.020156:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346050304:12345-0@lo:103:ldlm_bl.0 Request processed in 113us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826251.020162:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10936 00000100:00100000:1.0:1787826251.020395:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346050304:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826251.026225:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826251.026359:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332478208 00000100:00100000:2.0:1787826251.026369:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332478208 00000100:00100000:2.0:1787826251.026428:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68095 00000100:00100000:2.0:1787826251.026438:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222733:x1874663332478208:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826251.026457:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826251.026477:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48a17 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800bce expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.026539:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48a1e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.026553:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48a1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.026564:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48a1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826251.026621:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826251.026625:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826251.026646:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48a1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.026670:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48a17 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bce expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.026688:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48a1e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bce expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.026697:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826251.026700:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826251.026703:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826251.026758:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222733:x1874663332478208:12345-192.168.203.56@tcp:101:bash.0 Request processed in 321us (545us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.026765:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68095 00000100:00100000:0.0:1787826251.031928:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826251.033849:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346050432 00000100:00100000:0.0:1787826251.033857:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346050432 00000100:00100000:0.0:1787826251.034004:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7774 00000100:00100000:0.0:1787826251.034014:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346050432:12345-0@lo:104: 00010000:00010000:0.0:1787826251.034027:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.034145:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826251.034151:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346050432:12345-0@lo:104: Request processed in 139us (455us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826251.034158:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7774 00000100:00100000:2.0:1787826251.034238:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750e00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346050432:0@lo:104: 00010000:00010000:2.0:1787826251.034266:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.035154:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826251.035164:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.035174:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826251.035181:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826251.035190:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.035236:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.035244:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826251.035277:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.035284:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f489ed lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f489f4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.035308:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826251.035310:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826251.035312:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826251.035361:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826251.035379:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346050560:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826251.035431:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826251.035661:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346050560 00000100:00100000:0.0:1787826251.035669:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346050560 00010000:00010000:0.0:1787826251.035698:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f489f4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f489ed expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826251.035735:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21789 00000100:00100000:0.0:1787826251.035743:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346050560:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826251.035756:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787826251.038512:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957262 is committed 00010000:00010000:0.0:1787826251.038648:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f489f4 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f489ed expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.038674:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845784600) 00010000:00010000:0.0:1787826251.038680:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.038689:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.038696:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.038730:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826251.038815:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346050560:12345-0@lo:103:ldlm_bl.0 Request processed in 3071us (3382us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826251.038825:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21789 00010000:00010000:2.0:1787826251.039010:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.039022:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826251.039186:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-mdtlov_UUID:35095:1874663346050688:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826251.039229:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826251.039261:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f59c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826251.039757:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346050688 00000100:00100000:0.0:1787826251.039764:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346050688 00000100:00100000:0.0:1787826251.039826:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23065 00000100:00100000:0.0:1787826251.039837:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:35095:x1874663346050688:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826251.040051:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:35095:x1874663346050688:12345-0@lo:1000:mdt00_012.0 Request processed in 216us (821us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826251.040058:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23065 00000100:00100000:2.0:1787826251.040218:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-mdtlov_UUID:35095:1874663346050688:0@lo:1000:mdt00_012.0 00010000:00010000:2.0:1787826251.040269:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48a3a lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040282:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48a3a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040292:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48a3a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826251.040318:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48a3a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040330:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48a2c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040468:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48a2c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040475:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48a2c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040483:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48a2c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040493:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826251.040497:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48a2c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040505:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48a2c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040519:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826251.040523:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826251.040527:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48a2c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040533:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48a2c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040548:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040554:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040560:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040565:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826251.040568:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040574:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040584:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845784600) 00010000:00010000:2.0:1787826251.040650:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826251.040654:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040660:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f48a33 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040701:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f48a25 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bd5 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040716:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48a3a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bd5 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.040724:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f79462e8400) 00010000:00010000:2.0:1787826251.040727:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826251.040729:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826251.040820:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222733:x1874663332478336:12345-192.168.203.56@tcp:101:bash.0 Request processed in 8604us (8901us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.040828:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68096 00000100:00100000:0.0:1787826251.045843:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826251.046009:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332478464 00000100:00100000:2.0:1787826251.046018:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332478464 00000100:00100000:2.0:1787826251.046232:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68097 00000100:00100000:2.0:1787826251.046247:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222733:x1874663332478464:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826251.046271:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826251.046293:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f48a41 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800bdc expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.046368:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48a48 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.046380:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48a48 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.046392:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48a48 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826251.046492:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3a:0x0]: returning size 4096 00000004:00200000:2.0:1787826251.046497:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826251.046532:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f48a41 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bdc expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.046550:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48a48 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bdc expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.046559:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:2.0:1787826251.046562:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826251.046565:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826251.046633:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222733:x1874663332478464:12345-192.168.203.56@tcp:101:bash.0 Request processed in 392us (799us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.046640:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68097 00000100:00100000:0.0:1787826251.060083:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826251.060315:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332478592 00000100:00100000:2.0:1787826251.060326:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332478592 00000100:00100000:2.0:1787826251.060870:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68098 00000100:00100000:2.0:1787826251.060882:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222733:x1874663332478592:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826251.060915:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826251.060937:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f48a4f lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800be3 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.060993:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48a56 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.061006:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48a56 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.061017:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48a56 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826251.061063:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18b:0x0]: returning size 4096 00000004:00200000:2.0:1787826251.061078:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826251.061134:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f48a4f lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800be3 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.061153:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48a56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800be3 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.061162:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:2.0:1787826251.061165:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826251.061170:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826251.061247:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222733:x1874663332478592:12345-192.168.203.56@tcp:101:bash.0 Request processed in 367us (1182us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.061255:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68098 00000100:00100000:0.0:1787826251.071313:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826251.071440:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332478720 00000100:00100000:2.0:1787826251.071450:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332478720 00000100:00100000:2.0:1787826251.071513:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68099 00000100:00100000:2.0:1787826251.071523:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222733:x1874663332478720:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826251.071544:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826251.071563:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f48a5d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800bea expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.071635:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48a64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.071650:0:24703:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f48a10 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48a09 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.071659:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48a64 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.071691:0:24703:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f48a10 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f48a09 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826251.071707:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346050816:0@lo:104: 00000100:00100000:2.0:1787826251.071891:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826251.071945:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826251.072125:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346050816 00000100:00100000:0.0:1787826251.072132:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346050816 00000100:00100000:0.0:1787826251.072176:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7775 00000100:00100000:0.0:1787826251.072186:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346050816:12345-0@lo:104: 00010000:00010000:0.0:1787826251.072201:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f48a09 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f48a10 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.072272:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826251.072279:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346050816:12345-0@lo:104: Request processed in 96us (389us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826251.072289:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7775 00000100:00100000:2.0:1787826251.072560:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826251.072573:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b753100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346050944:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826251.072615:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826251.072667:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346050816:0@lo:104: 00010000:00010000:2.0:1787826251.072686:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48a64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.072746:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826251.072750:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826251.072751:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826251.074051:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346050944:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826251.074133:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48a64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.074143:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48a64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826251.074200:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x189:0x0]: returning size 4096 00000004:00200000:0.0:1787826251.074206:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826251.074250:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f48a5d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bea expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.074268:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48a64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bea expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.074277:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x0 (ffff9f7845784b00) 00010000:00010000:0.0:1787826251.074279:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826251.074282:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:0.0:1787826251.074342:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222733:x1874663332478720:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2821us (3039us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826251.074353:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68099 00000100:00100000:0.0:1787826251.079845:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826251.082138:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68100 00000100:00100000:0.0:1787826251.082163:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222733:x1874663332478976:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826251.082187:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826251.082190:0:6527:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826251.082221:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48a56 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800be3 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.082242:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:0.0:1787826251.082248:0:6527:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826251.082262:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48a6b lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800bf1 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.082390:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.082405:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.082416:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.082431:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.082438:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.082449:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826251.082576:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98e680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346051072:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826251.082721:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826251.082762:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826251.082837:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346051072 00000100:00100000:2.0:1787826251.082844:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346051072 00000100:00100000:2.0:1787826251.082891:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23066 00000100:00100000:2.0:1787826251.082899:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346051072:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826251.082978:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346051072:12345-0@lo:1000:mdt00_000.0 Request processed in 80us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.082985:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23066 00000004:00080000:0.0:1787826251.088318:0:6527:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4c8:0x0], index 0: 0 00010000:00010000:0.0:1787826251.088485:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48a80 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088501:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48a80 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088513:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48a80 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088549:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088556:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088563:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088570:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088576:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826251.088580:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088588:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088602:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:0.0:1787826251.088605:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826251.088609:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088616:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f48a72 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826251.088630:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cea81d12 reply state 000000002cdbbb6f transno 4294985287 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088825:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088834:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x5a734ce9 (ffff9f7845784400) 00010000:00010000:0.0:1787826251.088837:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826251.088841:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826251.088880:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48a6b lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bf1 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088896:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48a80 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bf1 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.088904:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3b:0x0].0x0 (ffff9f7845784700) 00010000:00010000:0.0:1787826251.088906:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826251.088909:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826251.089029:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:222733:x1874663332478976:12345-192.168.203.56@tcp:101:bash.0 Request processed in 6877us (9193us total) trans 4294985287 rc 0/0 00000100:00100000:0.0:1787826251.089039:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68100 00000100:00100000:0.0:1787826251.093654:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:0.0:1787826251.094195:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826251.101162:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826251.120748:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826251.127881:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:0.0:1787826251.128120:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826251.140683:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826251.159599:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826251.159638:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826251.160454:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826251.161807:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332480128 00000100:00100000:2.0:1787826251.161820:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332480128 00000100:00100000:2.0:1787826251.162179:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332480256 00000100:00100000:2.0:1787826251.162184:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332480256 00000100:00100000:2.0:1787826251.162220:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332480384 00000100:00100000:2.0:1787826251.162223:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332480384 00000100:00100000:2.0:1787826251.162252:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10937 00000100:00100000:2.0:1787826251.162262:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:15673:x1874663332480128:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826251.162364:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826251.162391:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f48aaa lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bff expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.162411:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:2.0:1787826251.162428:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48a1e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bce expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.162438:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826251.162446:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48a3a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bd5 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.162453:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f79462e8400) 00010000:00010000:2.0:1787826251.162460:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826251.162531:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:15673:x1874663332480128:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 270us (2941us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.162539:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10937 00000100:00100000:2.0:1787826251.162560:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10938 00000100:00100000:2.0:1787826251.162567:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:49149:x1874663332480256:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826251.162575:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826251.162586:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f488a4 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1db0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b81 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.162597:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db0:0x0].0x0 (ffff9f794b57a100) 00010000:00010000:2.0:1787826251.162604:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826251.162623:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:49149:x1874663332480256:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 57us (2985us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.162630:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10938 00000100:00100000:2.0:1787826251.162643:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10939 00000100:00100000:2.0:1787826251.162649:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:34881:x1874663332480384:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826251.162657:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826251.162669:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48a80 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bf1 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.162681:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3b:0x0].0x0 (ffff9f7845784700) 00010000:00010000:2.0:1787826251.162694:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48a48 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800bdc expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.162702:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:2.0:1787826251.162709:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48803 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e38:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b5e expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.162716:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e38:0x0].0x0 (ffff9f7843326200) 00010000:00010000:2.0:1787826251.162722:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826251.162741:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:34881:x1874663332480384:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 92us (2291us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826251.162748:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10939 00000100:00080000:2.0:1787826251.162770:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787826251.948552:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826252.633207:0:164102:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:1.0:1787826252.633345:0:164102:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:1.0:1787826252.973867:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826253.027745:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826253.027796:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826253.027887:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826253.027910:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826253.027985:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332480640 00000100:00100000:2.0:1787826253.027997:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332480640 00000100:00100000:2.0:1787826253.028075:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332481152 00000100:00100000:2.0:1787826253.028078:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332481152 00000100:00100000:2.0:1787826253.028125:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14181 00000100:00100000:2.0:1787826253.028137:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332480640:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826253.028213:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332480640:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 79us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826253.028220:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14181 00000100:00100000:2.0:1787826253.028239:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14182 00000100:00100000:2.0:1787826253.028246:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332481152:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826253.028275:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332481152:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (389us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826253.028281:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14182 00000100:00080000:2.0:1787826253.028299:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826253.037349:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826253.037741:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826253.037769:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332480768 00000100:00100000:0.0:1787826253.037779:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332480768 00000100:00100000:0.0:1787826253.037849:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14183 00000100:00100000:0.0:1787826253.037860:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332480768:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826253.037937:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332480768:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 80us (197us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826253.037945:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14183 00000100:00100000:1.0:1787826253.039102:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332480896 00000100:00100000:1.0:1787826253.039122:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332480896 00000100:00100000:0.0:1787826253.039268:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826253.043085:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826253.043142:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826253.048213:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332481408 00000100:00100000:2.0:1787826253.048226:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332481408 00000100:00100000:2.0:1787826253.048260:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332480512 00000100:00100000:2.0:1787826253.048263:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332480512 00000100:00100000:2.0:1787826253.048315:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2198 00000100:00100000:2.0:1787826253.048326:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d60450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332480512:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826253.048697:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d60450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332480512:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 371us (11650us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826253.048705:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2198 00000100:00080000:2.0:1787826253.051081:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826253.051163:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826253.051219:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332481024 00000100:00100000:2.0:1787826253.051229:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332481024 00000100:00100000:2.0:1787826253.051278:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8727 00000100:00100000:2.0:1787826253.051289:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332480896:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826253.051586:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332480896:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 286us (23776us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826253.051595:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8727 00000100:00100000:2.0:1787826253.051615:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8728 00000100:00100000:2.0:1787826253.051622:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332481024:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826253.051650:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332481024:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (12386us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826253.051656:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8728 00000100:00080000:2.0:1787826253.051761:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826253.051781:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826253.051804:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8729 00000100:00100000:2.0:1787826253.051811:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332481408:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826253.051842:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332481408:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 31us (23932us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826253.051848:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8729 00000100:00100000:1.0:1787826253.059754:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332481536 00000100:00100000:1.0:1787826253.059770:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332481536 00000100:00100000:1.0:1787826253.059855:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8730 00000100:00100000:1.0:1787826253.059866:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332481536:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826253.059944:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332481536:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 79us (16801us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826253.059951:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8730 00000100:00080000:1.0:1787826253.060091:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826253.806638:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346051328:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.806675:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826253.806698:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346051840:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.806719:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826253.806736:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346052352:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.806755:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826253.806771:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:0.0:1787826253.806778:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346051200:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.806802:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826253.806824:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346051712:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.806844:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826253.806861:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6baa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346052224:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.806880:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826253.806898:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346052736:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.806915:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826253.806942:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346051712 00000100:00100000:0.0:1787826253.806947:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346051712 00000100:00100000:0.0:1787826253.806982:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346052736 00000100:00100000:0.0:1787826253.806985:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346052736 00000100:00100000:0.0:1787826253.807007:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14189 00000100:00100000:0.0:1787826253.807014:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346051712:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807067:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346051712:12345-0@lo:400:kworker.0 Request processed in 54us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826253.807073:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14189 00000100:00100000:0.0:1787826253.807088:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14190 00000100:00100000:0.0:1787826253.807094:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346052736:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807143:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346052736:12345-0@lo:400:kworker.0 Request processed in 50us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826253.807148:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14190 00000100:00100000:0.0:1787826253.807175:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346051328 00000100:00100000:0.0:1787826253.807178:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346051328 00000100:00100000:0.0:1787826253.807200:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346052352 00000100:00100000:0.0:1787826253.807203:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346052352 00000100:00100000:0.0:1787826253.807234:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23067 00000100:00100000:0.0:1787826253.807239:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346051328:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807266:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346051328:12345-0@lo:400:kworker.0 Request processed in 27us (592us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826253.807271:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23067 00000100:00100000:0.0:1787826253.807281:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23068 00000100:00100000:0.0:1787826253.807287:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346052352:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807310:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346052352:12345-0@lo:400:kworker.0 Request processed in 23us (554us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826253.807315:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23068 00000100:00100000:0.0:1787826253.807330:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346051200 00000100:00100000:0.0:1787826253.807333:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346051200 00000100:00100000:0.0:1787826253.807357:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2199 00000100:00100000:0.0:1787826253.807363:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d60850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346051200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807390:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d60850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346051200:12345-0@lo:400:kworker.0 Request processed in 27us (586us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826253.807395:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2199 00000100:00100000:0.0:1787826253.807422:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346051840 00000100:00100000:0.0:1787826253.807427:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346051840 00000100:00100000:0.0:1787826253.807449:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346052224 00000100:00100000:0.0:1787826253.807453:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346052224 00000100:00100000:0.0:1787826253.807472:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8733 00000100:00100000:0.0:1787826253.807479:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346051840:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807506:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346051840:12345-0@lo:400:kworker.0 Request processed in 29us (786us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826253.807512:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8733 00000100:00100000:0.0:1787826253.807523:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8734 00000100:00100000:0.0:1787826253.807529:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346052224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807553:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346052224:12345-0@lo:400:kworker.0 Request processed in 25us (672us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826253.807558:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8734 00000100:00100000:0.0:1787826253.807606:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346051200:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807617:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346051712:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807626:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6baa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346052224:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807633:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346052736:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807641:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346051328:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807650:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346051840:0@lo:400:kworker.0 00000100:00100000:0.0:1787826253.807658:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346052352:0@lo:400:kworker.0 00000100:00080000:1.0:1787826253.807997:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826253.808002:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826253.808005:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826253.808007:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787826253.827241:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4893e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48945 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826253.827261:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4893e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48945 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.827307:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4893e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48945 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.827318:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826253.827344:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4893e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48945 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826253.827363:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985288 is committed 00000020:00080000:1.0:1787826253.827379:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784548d300 refcount 2 committed 0 result 0 batchid 3377 00000020:00080000:1.0:1787826253.827383:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f785042fb00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784548dcc0 00000020:00080000:1.0:1787826253.827386:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1479 00000020:00080000:1.0:1787826253.827388:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f785042f000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d006200 00000020:00080000:1.0:1787826253.827391:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1161 00000001:00080000:1.0:1787826253.827399:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985285 is committed 00000020:00080000:1.0:1787826253.827402:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7852f2a780 refcount 2 committed 0 result 0 batchid 3376 00000020:00080000:1.0:1787826253.827404:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f785042f500 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852f2a900 00000020:00080000:1.0:1787826253.827406:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1478 00000020:00080000:1.0:1787826253.827408:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f785042f300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78503d6800 00000020:00080000:1.0:1787826253.827410:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1160 00000004:00080000:1.0:1787826253.827416:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 396 committed 00000020:00080000:1.0:1787826253.827426:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784f6a4cc0 refcount 2 committed 0 result 0 batchid 3375 00000020:00080000:1.0:1787826253.827428:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f785042f900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f6a4300 00000020:00080000:1.0:1787826253.827431:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1477 00000020:00080000:1.0:1787826253.827432:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f785042f400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78503d6c00 00000020:00080000:1.0:1787826253.827435:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1159 00000020:00080000:1.0:1787826253.827442:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f797c1c0300 refcount 2 committed 0 result 0 batchid 3374 00000020:00080000:1.0:1787826253.827445:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7846fc9100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f797c1c0000 00000020:00080000:1.0:1787826253.827448:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1476 00000020:00080000:1.0:1787826253.827449:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7846fc9300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78503d6200 00000020:00080000:1.0:1787826253.827451:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1158 00000001:00080000:1.0:1787826253.827455:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985279 is committed 00000020:00080000:1.0:1787826253.827460:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3373 batchid updated 00000100:00100000:1.0:1787826253.827512:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346052864:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826253.827580:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:1.0:1787826253.827634:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3377 committed batchid 3373 00000004:00080000:1.0:1787826253.827747:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d02d3c0 with version 6748 00000020:00080000:1.0:1787826253.827758:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3366 cancel update log [0x200000401:0x3:0x0].1468: rc = 0 00000020:00080000:1.0:1787826253.827800:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3366 cancel update log [0x240000400:0x3:0x0].1149: rc = 0 00000004:00080000:1.0:1787826253.827824:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d02dc00 with version 6749 00000020:00080000:1.0:1787826253.827826:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3367 cancel update log [0x200000401:0x3:0x0].1469: rc = 0 00000020:00080000:1.0:1787826253.827845:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3367 cancel update log [0x240000400:0x3:0x0].1150: rc = 0 00000004:00080000:1.0:1787826253.827866:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78474d7780 with version 6750 00000020:00080000:1.0:1787826253.827869:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3368 cancel update log [0x200000401:0x3:0x0].1470: rc = 0 00000020:00080000:1.0:1787826253.827889:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3368 cancel update log [0x240000400:0x3:0x0].1151: rc = 0 00000004:00080000:1.0:1787826253.827907:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7845785180 with version 6751 00000020:00080000:1.0:1787826253.827910:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3369 cancel update log [0x200000401:0x3:0x0].1471: rc = 0 00000020:00080000:1.0:1787826253.827927:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3369 cancel update log [0x240000400:0x3:0x0].1152: rc = 0 00000004:00080000:1.0:1787826253.827947:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78457853c0 with version 6752 00000020:00080000:1.0:1787826253.827950:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3370 cancel update log [0x200000401:0x3:0x0].1472: rc = 0 00000020:00080000:1.0:1787826253.827966:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3370 cancel update log [0x240000400:0x3:0x0].1153: rc = 0 00000004:00080000:1.0:1787826253.827985:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7845785a80 with version 6753 00000020:00080000:1.0:1787826253.827988:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3371 cancel update log [0x200000401:0x3:0x0].1473: rc = 0 00000020:00080000:1.0:1787826253.828007:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3371 cancel update log [0x240000400:0x3:0x0].1154: rc = 0 00000004:00080000:1.0:1787826253.828027:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784adfc780 with version 6754 00000020:00080000:1.0:1787826253.828031:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3372 cancel update log [0x200000401:0x3:0x0].1474: rc = 0 00000020:00080000:1.0:1787826253.828067:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3372 cancel update log [0x240000400:0x3:0x0].1155: rc = 0 00000004:00080000:1.0:1787826253.828088:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784adfc300 with version 6755 00000020:00080000:1.0:1787826253.828091:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3373 cancel update log [0x200000401:0x3:0x0].1475: rc = 0 00000020:00080000:1.0:1787826253.828140:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3373 cancel update log [0x240000400:0x3:0x0].1157: rc = 0 00000020:00080000:1.0:1787826253.828146:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3373 00000100:00100000:1.0:1787826253.828231:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346052864 00000100:00100000:1.0:1787826253.828238:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346052864 00000100:00100000:1.0:1787826253.828287:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10940 00000100:00100000:1.0:1787826253.828296:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b470c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346052864:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826253.828309:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826253.828331:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f48945 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x188:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4893e expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.828346:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x188:0x0].0x5a734ce9 (ffff9f7961d81500) 00010000:00010000:1.0:1787826253.828358:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826253.828403:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b470c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346052864:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 107us (821us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826253.828411:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10940 00000004:00080000:1.0:1787826253.828464:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6748 rpc_version 6748 00000020:00100000:1.0:1787826253.828483:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826253.828490:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7854624a80 x1874663346053248/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826253.828511:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346053248:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.828541:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826253.828557:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826253.828571:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346052992:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1787826253.828591:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826253.828624:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346053248 00000100:00100000:1.0:1787826253.828628:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346053248 00000100:00100000:1.0:1787826253.828652:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23069 00000100:00100000:1.0:1787826253.828659:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346053248:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.828766:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346053248:12345-0@lo:1000:osp_up0-1.0 Request processed in 107us (224us total) trans 8589957265 rc 0/0 00000100:00100000:1.0:1787826253.828773:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23069 00000100:00100000:1.0:1787826253.828789:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346052992 00000100:00100000:1.0:1787826253.828792:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346052992 00000100:00100000:1.0:1787826253.828816:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30750 00000100:00100000:1.0:1787826253.828822:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346052992:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:1.0:1787826253.828838:0:20167:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1221 count 1 00010000:00010000:1.0:1787826253.828858:0:20167:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005bee640c/0xe21309aa81f48abf lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.828875:0:20167:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000975d3a2a/0xe21309aa81f4875b lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800b1f expref: 8 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.828884:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4c5:0x0].0x0 since=10s v=(0 0) 00010000:00010000:1.0:1787826253.828888:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826253.828893:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005bee640c/0xe21309aa81f48abf lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.828913:0:20167:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000975d3a2a/0xe21309aa81f4875b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b1f expref: 8 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.828925:0:20167:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000975d3a2a/0xe21309aa81f4875b lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800b1f expref: 8 pid: 24218 timeout: 8166 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826253.828934:0:20167:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20167:1874663346053376:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826253.828984:0:20167:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69c600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826253.829037:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346053248:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826253.829045:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6749 rpc_version 6749 00000020:00100000:1.0:1787826253.829074:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826253.829079:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f79785e5500 x1874663346053632/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826253.829091:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346053632:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.829137:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826253.829153:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826253.829165:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346053632 00000100:00100000:1.0:1787826253.829169:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346053632 00000100:00100000:1.0:1787826253.829194:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23070 00000100:00100000:1.0:1787826253.829201:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346053632:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.829274:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346053632:12345-0@lo:1000:osp_up0-1.0 Request processed in 74us (137us total) trans 8589957266 rc 0/0 00000100:00100000:1.0:1787826253.829280:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23070 00000100:00100000:1.0:1787826253.829313:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346052864:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826253.829341:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346053632:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826253.829348:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6750 rpc_version 6750 00000020:00100000:1.0:1787826253.829358:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826253.829363:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f79785e7100 x1874663346053888/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826253.829373:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346053888:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.829399:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826253.829414:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826253.829425:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346053888 00000100:00100000:1.0:1787826253.829428:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346053888 00000100:00100000:1.0:1787826253.829449:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23071 00000100:00100000:1.0:1787826253.829456:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346053888:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.829522:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346053888:12345-0@lo:1000:osp_up0-1.0 Request processed in 66us (122us total) trans 8589957267 rc 0/0 00000100:00100000:1.0:1787826253.829529:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23071 00000100:00100000:1.0:1787826253.829556:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346053888:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826253.829564:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6751 rpc_version 6751 00000020:00100000:1.0:1787826253.829576:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826253.829581:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc32680 x1874663346054144/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826253.829593:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346054144:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.829618:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826253.829632:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826253.829643:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346054144 00000100:00100000:1.0:1787826253.829646:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346054144 00000100:00100000:1.0:1787826253.829666:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23072 00000100:00100000:1.0:1787826253.829671:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346054144:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.829736:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346054144:12345-0@lo:1000:osp_up0-1.0 Request processed in 63us (116us total) trans 8589957268 rc 0/0 00000100:00100000:1.0:1787826253.829741:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23072 00000100:00100000:1.0:1787826253.829769:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346054144:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826253.829775:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6752 rpc_version 6752 00000020:00100000:1.0:1787826253.829786:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826253.829790:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc33480 x1874663346054400/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826253.829801:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346054400:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.829825:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826253.829838:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631ab40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826253.829847:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346054400 00000100:00100000:1.0:1787826253.829850:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346054400 00000100:00100000:1.0:1787826253.829868:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23073 00000100:00100000:1.0:1787826253.829875:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346054400:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.829929:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346054400:12345-0@lo:1000:osp_up0-1.0 Request processed in 54us (103us total) trans 8589957269 rc 0/0 00000100:00100000:1.0:1787826253.829935:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23073 00000100:00100000:1.0:1787826253.829957:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346054400:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826253.829963:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6753 rpc_version 6753 00000020:00100000:1.0:1787826253.829972:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826253.829977:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc31500 x1874663346054656/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826253.829988:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346054656:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.830007:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826253.830021:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826253.830029:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346054656 00000100:00100000:1.0:1787826253.830033:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346054656 00010000:00010000:2.0:1787826253.831100:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f489fb lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831128:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f489fb lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831136:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f489fb lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831142:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826253.831148:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f489fb lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831156:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f489fb lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831175:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:2.0:1787826253.831179:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826253.831183:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f489fb lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831190:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f489fb lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831233:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48a02 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831240:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48a02 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831247:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48a02 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831253:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826253.831256:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48a02 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831262:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48a02 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831270:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:2.0:1787826253.831272:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826253.831275:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48a02 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831281:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48a02 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831300:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831306:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831312:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831318:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826253.831321:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831327:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831335:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x5a734ce9 (ffff9f7845784400) 00010000:00010000:2.0:1787826253.831337:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826253.831340:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.831345:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48a79 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826253.836078:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23074 00000100:00100000:1.0:1787826253.836100:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346054656:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.836339:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346054656:12345-0@lo:1000:osp_up0-1.0 Request processed in 248us (6330us total) trans 8589957270 rc 0/0 00000100:00100000:1.0:1787826253.836347:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23074 00000100:00100000:1.0:1787826253.840241:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346054912 00000100:00100000:1.0:1787826253.840251:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346054912 00000100:00100000:1.0:1787826253.840317:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23075 00000100:00100000:1.0:1787826253.840328:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346054912:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.840615:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346054912:12345-0@lo:1000:osp_up0-1.0 Request processed in 287us (1325us total) trans 8589957271 rc 0/0 00000100:00100000:1.0:1787826253.840623:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23075 00000100:00100000:1.0:1787826253.840672:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346054912:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826253.840684:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6755 rpc_version 6755 00000020:00100000:1.0:1787826253.840702:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826253.840708:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d8bf100 x1874663346055168/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826253.840724:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346055168:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826253.840756:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826253.840783:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826253.840828:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346055168 00000100:00100000:0.0:1787826253.840832:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346055168 00000100:00100000:0.0:1787826253.840864:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23076 00000100:00100000:0.0:1787826253.840871:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346055168:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826253.840965:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346055168:12345-0@lo:1000:osp_up0-1.0 Request processed in 94us (208us total) trans 8589957272 rc 0/0 00000100:00100000:0.0:1787826253.840972:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23076 00000100:00100000:0.0:1787826253.843121:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826253.843357:0:20167:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20167:1874663346053376:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826253.843383:0:20167:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005bee640c/0xe21309aa81f48abf lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.844153:0:20167:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005bee640c/0xe21309aa81f48abf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.844164:0:20167:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005bee640c/0xe21309aa81f48abf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.844173:0:20167:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005bee640c/0xe21309aa81f48abf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.844181:0:20167:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005bee640c/0xe21309aa81f48abf lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826253.844200:0:20167:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826253.844366:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346052992:12345-0@lo:6:osp-syn-1-1.0 Request processed in 15544us (15773us total) trans 4294976137 rc 0/0 00000100:00100000:1.0:1787826253.844374:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30750 00000004:00080000:1.0:1787826253.844411:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7854626d80/2, rc 0, transno 4294976137 00000100:00100000:1.0:1787826253.844417:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346052992:0@lo:6:osp-syn-1-1.0 00010000:00010000:1.0:1787826253.998347:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826254.061027:0:8450:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332481792:12345-192.168.203.56@tcp:4:bash.0 Request processed in 5856us (6187us total) trans 4294973829 rc 0/0 00000100:00100000:1.0:1787826254.061041:0:8450:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27621 00002000:00100000:1.0:1787826254.061064:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4c8:0x0] (0->5) req@ffff9f784c6baa00 x1874663332481792/t4294973829(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:610/0 lens 496/448 e 0 to 0 dl 1787826265 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826254.061082:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784c6baa00 with x1874663332481792 ext(0->5) 00002000:00010000:1.0:1787826254.061089:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784c6baa00 00000100:00100000:0.0:1787826254.064092:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826254.064163:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1787826254.079460:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976138 is committed 00000001:00080000:0.0:1787826254.079477:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976137 is committed 00000100:00100000:2.0:1787826254.079816:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332482176:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 13986us (15645us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826254.079828:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30752 00000100:00100000:1.0:1787826254.316480:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346055296:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826254.316553:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826254.316961:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976137 gen 2 00000004:00080000:1.0:1787826254.316967:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7854626d80, transno 4294976137 00000100:00100000:1.0:1787826254.316994:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346055296:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826254.830438:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346055680:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826254.830480:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826254.830934:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346055680 00000100:00100000:0.0:1787826254.830940:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346055680 00000100:00100000:0.0:1787826254.830982:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23077 00000100:00100000:0.0:1787826254.830992:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346055680:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1787826254.831022:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443446784 avail 1297207296 00000100:00100000:0.0:1787826254.831083:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346055680:12345-0@lo:41:osp-pre-1-0.0 Request processed in 92us (601us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826254.831094:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23077 00000100:00100000:0.0:1787826254.831142:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346055552 00000100:00100000:0.0:1787826254.831147:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346055552 00000100:00100000:0.0:1787826254.831178:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23078 00000100:00100000:0.0:1787826254.831189:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:3655:x1874663346055552:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787826254.831213:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442840576 avail 1296601088 00000100:00100000:0.0:1787826254.831255:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:3655:x1874663346055552:12345-0@lo:41:osp-pre-0-1.0 Request processed in 68us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826254.831263:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23078 00000100:00100000:1.0:1787826254.831545:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f685500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346055808:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826254.831653:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985279 gen 2 00000004:00080000:2.0:1787826254.831659:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985278 committed transno 4294985279 00000020:00080000:2.0:1787826254.831665:0:3652:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7851ff5000 refcount 2 committed 0 result 0 batchid 4294979072 00000020:00080000:2.0:1787826254.831669:0:3652:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2de300 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7851ff5840 00000020:00080000:2.0:1787826254.831673:0:3652:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].928 00000020:00080000:2.0:1787826254.831675:0:3652:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2de800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d036200 00000020:00080000:2.0:1787826254.831678:0:3652:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].925 00000004:00080000:2.0:1787826254.831711:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985279 committed transno 4294985279 00000100:00100000:2.0:1787826254.831736:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346055680:0@lo:41:osp-pre-1-0.0 00000020:00080000:0.0:1787826254.831783:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979072 committed batchid 4294979071 00000020:00080000:0.0:1787826254.831909:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979071 00000100:00100000:2.0:1787826254.832818:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b661f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346055552:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826254.837565:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346055936:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826254.838279:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826254.838727:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f685500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346055808:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1787826255.020360:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826255.294946:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826255.298134:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332482304 00000100:00100000:2.0:1787826255.298148:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332482304 00000100:00100000:2.0:1787826255.298224:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68102 00000100:00100000:2.0:1787826255.298235:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:222914:x1874663332482304:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826255.298254:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826255.298276:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48ac6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c0d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.298337:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.298350:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.298360:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826255.298406:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826255.298410:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826255.298431:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.298456:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48ac6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c0d expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.298478:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c0d expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.298486:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826255.298489:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826255.298493:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00080000:1.0:1787826255.298652:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826255.298825:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:222914:x1874663332482304:12345-192.168.203.56@tcp:101:cat.0 Request processed in 590us (3884us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826255.298839:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68102 00000100:00100000:1.0:1787826255.308758:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826255.312438:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332482560 00000100:00100000:0.0:1787826255.312448:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332482560 00000100:00100000:0.0:1787826255.312515:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68104 00000100:00100000:0.0:1787826255.312527:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:222914:x1874663332482560:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826255.312556:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826255.312577:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f48af0 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c1b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.312633:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f48af7 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.312646:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f48af7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.312658:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f48af7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826255.312749:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3a:0x0]: returning size 4096 00000004:00200000:0.0:1787826255.312754:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826255.312789:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f48af0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c1b expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.312806:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f48af7 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c1b expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.312815:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826255.312818:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826255.312822:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826255.312875:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:222914:x1874663332482560:12345-192.168.203.56@tcp:101:cat.0 Request processed in 351us (599us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.312882:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68104 00000100:00080000:1.0:1787826255.315965:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826255.317347:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826255.318212:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332482816 00000100:00100000:2.0:1787826255.318223:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332482816 00000100:00100000:2.0:1787826255.318281:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68106 00000100:00100000:2.0:1787826255.318292:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:222914:x1874663332482816:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826255.318311:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826255.318334:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48b0c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c29 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.318394:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48b13 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.318408:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48b13 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.318420:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48b13 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826255.318469:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x189:0x0]: returning size 4096 00000004:00200000:2.0:1787826255.318475:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826255.318510:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48b0c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c29 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.318525:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48b13 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c29 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.318532:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:2.0:1787826255.318535:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826255.318538:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826255.318699:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222914:x1874663332482816:12345-192.168.203.56@tcp:101:cat.0 Request processed in 408us (1358us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826255.318706:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68106 00000100:00100000:0.0:1787826255.323926:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826255.324003:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332483072 00000100:00100000:2.0:1787826255.324013:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332483072 00000100:00100000:2.0:1787826255.324084:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68107 00000100:00100000:2.0:1787826255.324095:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:222914:x1874663332483072:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826255.324145:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826255.324170:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48b1a lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c30 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.324247:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.324260:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.324271:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.324290:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.324297:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.324304:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00020000:2.0:1787826255.324384:0:35095:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000100:00100000:1.0:1787826255.827150:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826255.827261:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332483200 00000100:00100000:0.0:1787826255.827271:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332483200 00000100:00100000:0.0:1787826255.827355:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68108 00000100:00100000:0.0:1787826255.827366:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222916:x1874663332483200:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826255.827385:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826255.827407:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f48b2f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c37 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.827473:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.827488:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.827502:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826255.827547:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826255.827551:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826255.827570:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.827593:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f48b2f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c37 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.827617:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c37 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.827628:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826255.827631:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826255.827635:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826255.827711:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222916:x1874663332483200:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 347us (605us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.827719:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68108 00000100:00080000:1.0:1787826255.827867:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826255.835637:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826255.835782:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332483328 00000100:00100000:2.0:1787826255.835794:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332483328 00000100:00100000:2.0:1787826255.835860:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68109 00000100:00100000:2.0:1787826255.835871:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222916:x1874663332483328:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826255.835894:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826255.835914:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48b3d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c3e expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.835978:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.835992:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836003:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836020:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836032:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836050:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836146:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836156:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836162:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826255.836184:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836197:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836203:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836209:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836215:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836220:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826255.836225:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836233:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836245:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826255.836247:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826255.836252:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836258:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48b44 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836270:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836276:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836281:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836285:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826255.836288:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836293:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836300:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79462e8400) 00010000:00010000:2.0:1787826255.836302:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826255.836304:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836313:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f48b4b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836342:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48b3d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c3e expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836355:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c3e expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.836361:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f7843326e00) 00010000:00010000:2.0:1787826255.836363:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826255.836365:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826255.836429:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:222916:x1874663332483328:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 560us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826255.836436:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68109 00000100:00100000:2.0:1787826255.845538:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826255.845640:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332483456 00000100:00100000:1.0:1787826255.845649:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332483456 00000100:00100000:1.0:1787826255.845726:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68110 00000100:00100000:1.0:1787826255.845737:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:222916:x1874663332483456:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826255.845757:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826255.845779:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f48b59 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c45 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.845839:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f48b60 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.845852:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f48b60 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.845866:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f48b60 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826255.845958:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3a:0x0]: returning size 4096 00000004:00200000:1.0:1787826255.845963:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826255.845998:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f48b59 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c45 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.846016:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f48b60 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c45 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.846025:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:1.0:1787826255.846028:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826255.846032:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826255.846146:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222916:x1874663332483456:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 411us (621us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826255.846153:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68110 00000100:00080000:2.0:1787826255.846638:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826255.849986:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826255.863577:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000001:00080000:0.0:1787826255.865443:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973829 is committed 00000001:00080000:0.0:1787826255.865456:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973828 is committed 00000100:00100000:2.0:1787826255.880925:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826255.880971:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826255.881066:0:125380:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332484096 00000100:00100000:1.0:1787826255.881076:0:125380:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332484096 00000100:00100000:1.0:1787826255.881172:0:125380:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1220 00000100:00100000:1.0:1787826255.881183:0:125380:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222916:x1874663332484096:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:1.0:1787826255.881234:0:125380:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826255.881260:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f48b98 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c53 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.881278:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3b:0x0].0x0 (ffff9f7843326d00) 00010000:00010000:1.0:1787826255.881290:0:125380:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826255.881349:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056064:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.881398:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.881566:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.881911:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332483840 00000100:00100000:0.0:1787826255.881916:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332483840 00000100:00100000:0.0:1787826255.881959:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10941 00000100:00100000:0.0:1787826255.881967:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:2394:x1874663332483840:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:0.0:1787826255.881977:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826255.881992:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f48b7c lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c4c expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.882007:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7843326600) 00010000:00010000:0.0:1787826255.882014:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826255.882066:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2394:x1874663332483840:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 100us (1151us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.882077:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10941 00000100:00100000:1.0:1787826255.882412:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056064:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.882471:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056192:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.882499:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.882530:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.882623:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346056192 00000100:00100000:0.0:1787826255.882629:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346056192 00000100:00100000:0.0:1787826255.882667:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23080 00000100:00100000:0.0:1787826255.882675:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:125380:x1874663346056192:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.882757:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:125380:x1874663346056192:12345-0@lo:1000:mdt_io00_003.0 Request processed in 83us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.882769:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23080 00000100:00100000:2.0:1787826255.882833:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056192:0@lo:1000:mdt_io00_003.0 00000004:00080000:2.0:1787826255.882857:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x189:0x0] 00000100:00100000:2.0:1787826255.882905:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056320:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826255.882933:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826255.882958:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826255.883673:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056320:0@lo:1000:mdt_io00_003.0 00000004:00080000:2.0:1787826255.883707:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x189:0x0] 00000100:00100000:2.0:1787826255.883730:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056448:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826255.883757:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826255.883791:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.884151:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346056448 00000100:00100000:0.0:1787826255.884156:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346056448 00000100:00100000:0.0:1787826255.884192:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23082 00000100:00100000:0.0:1787826255.884200:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:125380:x1874663346056448:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.884271:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:125380:x1874663346056448:12345-0@lo:1000:mdt_io00_003.0 Request processed in 71us (512us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.884278:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23082 00000100:00100000:2.0:1787826255.884397:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056448:0@lo:1000:mdt_io00_003.0 00000004:00020000:2.0:1787826255.884428:0:125380:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00000004:00020000:0.0:1787826255.884472:0:35095:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4440 00010000:00010000:2.0:1787826255.884474:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.884493:0:125380:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48b05 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c22 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.884503:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.884531:0:125380:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48b05 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c22 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884534:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48ba6 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884551:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48ba6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884564:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48ba6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.884565:0:125380:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48b05 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c22 expref: 8 pid: 24703 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826255.884586:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3b:0x0]: returning size 6 00000100:00100000:2.0:1787826255.884588:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001_UUID:125380:1874663346056576:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826255.884656:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5540 going to sleep for 11 seconds 00010000:00010000:0.0:1787826255.884681:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884694:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884700:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884707:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884713:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826255.884718:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884726:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884738:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f7843326200) 00010000:00010000:0.0:1787826255.884741:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826255.884744:0:35095:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884753:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884759:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f48b21 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884772:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884778:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884784:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884789:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826255.884792:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884798:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884806:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x5a734ce9 (ffff9f79462e8300) 00010000:00010000:0.0:1787826255.884808:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826255.884811:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884817:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48b28 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884860:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48b1a lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c30 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884874:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48ba6 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c30 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.884881:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3b:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:0.0:1787826255.884884:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826255.884886:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826255.884909:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222914:x1874663332483072:12345-192.168.203.56@tcp:101:cat.0 Request processed in 560816us (560992us total) trans 4294985289 rc 0/0 00000100:00100000:0.0:1787826255.884919:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68107 00000100:00100000:2.0:1787826255.891708:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826255.891749:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826255.892073:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332484352 00000100:00100000:0.0:1787826255.892080:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332484352 00000100:00100000:0.0:1787826255.892146:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30753 00000100:00100000:0.0:1787826255.892158:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332484352:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826255.892172:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826255.892188:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f48bb4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c5a expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.892239:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346056704:192.168.203.56@tcp:106: 00000100:00100000:0.0:1787826255.892297:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb73c0 going to sleep for 11 seconds 00010000:00010000:1.0:1787826255.892516:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f48b05 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c22 expref: 8 pid: 24703 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826255.892532:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001_UUID:125380:1874663346056576:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826255.892552:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.892561:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.892567:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.892589:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.892599:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.892608:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.892648:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.892655:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.892665:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826255.892681:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346056832:0@lo:101:mdt_io00_003.0 00000100:00100000:1.0:1787826255.892715:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.892745:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.892807:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346056832 00000100:00100000:0.0:1787826255.892812:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346056832 00000100:00100000:0.0:1787826255.892850:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23083 00000100:00080000:2.0:1787826255.892853:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826255.892860:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:125380:x1874663346056832:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:0.0:1787826255.892872:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826255.892884:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48bc9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48bc2 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.892901:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48b13 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c29 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.892910:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48bc9 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48bc2 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.892931:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48b13 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c29 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.892943:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48b13 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c29 expref: 8 pid: 6527 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.892954:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346056960:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826255.892993:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826255.900697:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00002000:00010000:0.0:1787826255.901127:0:8759:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4c8:0x0] updating lvb size: 0 -> 6 00002000:00010000:0.0:1787826255.901132:0:8759:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4c8:0x0] updating lvb mtime: 0 -> 1787826250 00002000:00010000:0.0:1787826255.901135:0:8759:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4c8:0x0] updating lvb atime: 0 -> 1787826250 00002000:00010000:0.0:1787826255.901137:0:8759:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4c8:0x0] updating lvb ctime: 0 -> 1787826250 00002000:00010000:0.0:1787826255.901139:0:8759:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4c8:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:0.0:1787826255.901159:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346056704:192.168.203.56@tcp:106: 00010000:00010000:0.0:1787826255.901198:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f48bb4 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c5a expref: 6 pid: 8759 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.901221:0:8759:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f48bb4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c5a expref: 6 pid: 8759 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.901241:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826255.901293:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332484352:12345-192.168.203.56@tcp:101:cat.0 Request processed in 9138us (9542us total) trans 0 rc 301/301 00000100:00100000:0.0:1787826255.901302:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30753 00010000:00010000:1.0:1787826255.901323:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48b13 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c29 expref: 9 pid: 6527 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826255.901338:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346056960:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826255.901357:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48bc9 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48bc2 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.901365:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:1.0:1787826255.901369:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826255.901371:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826255.905062:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826255.905080:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346057088 00000100:00100000:0.0:1787826255.905089:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346057088 00000100:00100000:2.0:1787826255.905097:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826255.905176:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7776 00000100:00100000:0.0:1787826255.905186:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78437af850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346057088:12345-0@lo:105: 00010000:00010000:0.0:1787826255.905202:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.905213:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.905225:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.905236:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.905244:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.905253:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.905270:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecfe00) 00000100:00100000:0.0:1787826255.905327:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78437af850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346057088:12345-0@lo:105: Request processed in 138us (393us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.905339:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7776 00000100:00100000:0.0:1787826255.905425:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346057088:0@lo:105: 00010000:00010000:0.0:1787826255.905446:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826255.905490:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332484480:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 4435us (4800us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.905509:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21793 00000100:00100000:1.0:1787826255.907799:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+16:125380:x1874663346056832:12345-0@lo:101:mdt_io00_003.0 Request processed in 14930us (15072us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826255.907812:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23083 00000100:00100000:1.0:1787826255.908074:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346057344 00000100:00100000:1.0:1787826255.908082:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346057344 00000100:00100000:1.0:1787826255.908149:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23084 00000100:00100000:1.0:1787826255.908159:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+16:125380:x1874663346057344:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:1.0:1787826255.908175:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826255.908190:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f48bde lrc: 2/0,0 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48bd7 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.908210:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 2/0,1 mode: TXN/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.908219:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f48bde lrc: 4/0,0 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48bd7 expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.908234:0:6532:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00080000:00080000:1.0:1787826255.908265:0:6532:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00010000:00010000:1.0:1787826255.908320:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f48bde lrc: 3/0,0 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48bd7 expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.908329:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x4ec94fd6 (ffff9f79460a2700) 00010000:00010000:1.0:1787826255.908332:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826255.908335:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f48bde lrc: 3/0,0 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48bd7 expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.908342:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826255.908381:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+17:125380:x1874663346057344:12345-0@lo:101:mdt_io00_003.0 Request processed in 223us (388us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826255.908386:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23084 00000100:00100000:1.0:1787826255.908415:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346057344:0@lo:101:mdt_io00_003.0 00010000:00010000:1.0:1787826255.908433:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.908451:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.910296:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332484736 00000100:00100000:0.0:1787826255.910306:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332484736 00010000:00010000:0.0:1787826255.910355:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ddbcfe2a/0xe21309aa81f48a8e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bf8 expref: 6 pid: 8445 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.910388:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21794 00000100:00100000:0.0:1787826255.910399:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332484736:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826255.910412:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00080000:2.0:1787826255.910428:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787826255.910434:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ddbcfe2a/0xe21309aa81f48a8e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bf8 expref: 6 pid: 8445 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.910451:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ddbcfe2a/0xe21309aa81f48a8e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bf8 expref: 6 pid: 8445 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.910478:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826255.910483:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826255.910491:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.910500:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.910532:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.910550:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8459:1874663346057472:192.168.203.56@tcp:105: 00000100:00100000:0.0:1787826255.910627:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00010000:00010000:1.0:1787826255.912268:0:35097:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddbcfe2a/0xe21309aa81f48a8e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800bf8 expref: 6 pid: 8445 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826255.912287:0:35097:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:35097:1874663346057216:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826255.912314:0:35097:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.912326:0:35097:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:1.0:1787826255.912377:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:222914:x1874663332484608:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7057us (7281us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826255.912384:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30754 00000020:00080000:2.0:1787826255.914757:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979072 batchid updated 00000001:00080000:2.0:1787826255.914792:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957272 is committed 00000001:00080000:2.0:1787826255.914818:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957264 is committed 00000004:00080000:2.0:1787826255.915188:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1806 rpc_version 1806 00000020:00100000:2.0:1787826255.915221:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826255.915230:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977f28700 x1874663346057728/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826255.915252:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f28700 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346057728:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826255.915318:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826255.915359:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826255.915944:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f28700 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346057728:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826255.916131:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8459:1874663346057472:192.168.203.56@tcp:105: 00010000:00010000:0.0:1787826255.916153:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826255.916186:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.916190:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332484736:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 5793us (7090us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826255.916196:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.916199:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21794 00010000:00010000:1.0:1787826255.916203:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.916208:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826255.916214:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.916221:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.916237:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x4ec94fd6 (ffff9f79460a2700) 00010000:00010000:1.0:1787826255.916240:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826255.916245:0:3639:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f48bde lrc: 2/0,0 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48bd7 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.916256:0:3639:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f48bde lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48bd7 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.916264:0:3639:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f48bde lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48bd7 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.916290:0:3639:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f48bde lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48bd7 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826255.916300:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3639:1874663346057856:0@lo:105: 00000100:00100000:1.0:1787826255.916331:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.916365:0:3639:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.916476:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346057856 00000100:00100000:0.0:1787826255.916485:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346057856 00000100:00100000:0.0:1787826255.916528:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7777 00000100:00100000:0.0:1787826255.916538:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:3639:x1874663346057856:12345-0@lo:105: 00010000:00010000:0.0:1787826255.916549:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.916559:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.916567:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.916575:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 5/0,1 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.916582:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.916595:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.916619:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69200) 00000100:00100000:0.0:1787826255.916657:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:3639:x1874663346057856:12345-0@lo:105: Request processed in 122us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.916664:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7777 00000100:00100000:2.0:1787826255.917081:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3639:1874663346057856:0@lo:105: 00010000:00010000:2.0:1787826255.917099:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.917127:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f48aa3 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.917559:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.917570:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826255.917598:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x189:0x0] 00000100:00100000:1.0:1787826255.917646:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346057984:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.917686:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.917704:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.917995:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346057984 00000100:00100000:0.0:1787826255.918001:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346057984 00000100:00100000:0.0:1787826255.918048:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23086 00000100:00100000:0.0:1787826255.918056:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:125380:x1874663346057984:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.918155:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:125380:x1874663346057984:12345-0@lo:1000:mdt_io00_003.0 Request processed in 99us (467us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.918162:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23086 00000100:00100000:1.0:1787826255.918164:0:125380:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957272 gen 2 00000004:00080000:1.0:1787826255.918169:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957265 committed transno 8589957272 00000004:00080000:1.0:1787826255.918183:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957266 committed transno 8589957272 00000004:00080000:1.0:1787826255.918194:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957267 committed transno 8589957272 00000004:00080000:1.0:1787826255.918205:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957268 committed transno 8589957272 00000004:00080000:1.0:1787826255.918215:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957269 committed transno 8589957272 00000004:00080000:1.0:1787826255.918224:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957270 committed transno 8589957272 00000004:00080000:1.0:1787826255.918231:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957271 committed transno 8589957272 00000004:00080000:1.0:1787826255.918245:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957272 committed transno 8589957272 00000100:00100000:1.0:1787826255.918258:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346057984:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.918302:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afc000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058112:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.918325:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.918351:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826255.918500:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826255.921126:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afc000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058112:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.921199:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058240:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.921238:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826255.921274:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d82ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826255.921298:0:20151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332484864 00000100:00100000:1.0:1787826255.921313:0:20151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332484864 00002000:00100000:0.0:1787826255.921406:0:20151:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4c8:0x0] (0->4095) req@ffff9f7977f2aa00 x1874663332484864/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:611/0 lens 488/0 e 0 to 0 dl 1787826266 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826255.921417:0:20151:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977f2aa00 with x1874663332484864 ext(0->4095) 00002000:00010000:0.0:1787826255.921422:0:20151:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7977f2aa00 00000100:00100000:0.0:1787826255.921448:0:20151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27623 00000100:00100000:0.0:1787826255.921456:0:20151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332484864:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:0.0:1787826255.921604:0:20151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332484864:12345-192.168.203.56@tcp:3:cat.0 Request processed in 148us (3104us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787826255.921611:0:20151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27623 00002000:00100000:0.0:1787826255.921617:0:20151:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x4c8:0x0] (0->4095) req@ffff9f7977f2aa00 x1874663332484864/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:611/0 lens 488/4536 e 0 to 0 dl 1787826266 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826255.921624:0:20151:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977f2aa00 with x1874663332484864 ext(0->4095) 00002000:00010000:0.0:1787826255.921627:0:20151:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7977f2aa00 00000100:00100000:2.0:1787826255.921770:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346058240 00000100:00100000:2.0:1787826255.921775:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346058240 00000100:00100000:2.0:1787826255.921814:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23088 00000100:00100000:2.0:1787826255.921822:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:125380:x1874663346058240:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826255.921933:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:125380:x1874663346058240:12345-0@lo:1000:mdt_io00_003.0 Request processed in 112us (694us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826255.921940:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23088 00000100:00100000:0.0:1787826255.922004:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058240:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.922081:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058368:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.922148:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826255.922171:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.922818:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058368:0@lo:1000:mdt_io00_003.0 00010000:00010000:0.0:1787826255.922941:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.922952:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.922961:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.922973:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058496:0@lo:101:mdt_io00_003.0 00000100:00100000:0.0:1787826255.923005:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826255.923029:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.924987:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058496:0@lo:101:mdt_io00_003.0 00010000:00010000:0.0:1787826255.925014:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48bec expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.925027:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bec expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826255.925060:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000403:0x1db1:0x0] 00000100:00100000:0.0:1787826255.925141:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058624:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.926550:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826255.927080:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826255.927132:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826255.927210:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332485120 00000100:00100000:0.0:1787826255.927218:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332485120 00000100:00100000:0.0:1787826255.927276:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9277 00000100:00100000:0.0:1787826255.927286:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:222914:x1874663332485120:12345-192.168.203.56@tcp:35:cat.0 00000100:00100000:0.0:1787826255.927745:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222914:x1874663332485120:12345-192.168.203.56@tcp:35:cat.0 Request processed in 459us (640us total) trans 4294985291 rc 0/0 00000100:00100000:0.0:1787826255.927753:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9277 00000100:00100000:0.0:1787826255.936848:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058624:0@lo:1000:mdt_io00_003.0 00000004:00080000:0.0:1787826255.936901:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x189:0x0] 00010000:00010000:0.0:1787826255.936961:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.936982:0:125380:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48ba6 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c30 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.936992:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.937046:0:125380:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48ba6 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c30 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.937065:0:125380:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48ba6 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c30 expref: 8 pid: 35095 timeout: 8168 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.937080:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001_UUID:125380:1874663346058752:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826255.937219:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.941718:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001_UUID:125380:1874663346058752:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826255.941750:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826255.942025:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:0.0:1787826255.942520:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.942530:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.942592:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.942600:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x2 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.942611:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x2 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826255.942638:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058880:0@lo:101:mdt_io00_003.0 00000100:00100000:0.0:1787826255.942681:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826255.942719:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826255.942886:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346058880 00000100:00100000:2.0:1787826255.942892:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346058880 00000100:00100000:2.0:1787826255.942932:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23092 00000100:00100000:2.0:1787826255.942941:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:125380:x1874663346058880:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:2.0:1787826255.942958:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826255.942974:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48c01 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48bfa expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.942994:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48c01 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48bfa expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.943007:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48c01 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48bfa expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.943014:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f79462e8400) 00010000:00010000:2.0:1787826255.943016:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826255.943019:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826255.943074:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346058880:12345-0@lo:101:mdt_io00_003.0 Request processed in 134us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826255.943081:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23092 00000100:00100000:1.0:1787826255.943158:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346058880:0@lo:101:mdt_io00_003.0 00010000:00010000:1.0:1787826255.943173:0:125380:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 4/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.943183:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.943197:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826255.943277:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059008:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.943308:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.943329:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.943384:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346059008 00000100:00100000:0.0:1787826255.943389:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346059008 00000100:00100000:0.0:1787826255.943433:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23093 00000100:00100000:0.0:1787826255.943441:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346059008:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.943508:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346059008:12345-0@lo:1000:mdt_io00_003.0 Request processed in 66us (198us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826255.943515:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23093 00000100:00100000:1.0:1787826255.943627:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059008:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.943664:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059136:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.943688:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.943712:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826255.943799:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346059136 00000100:00100000:2.0:1787826255.943804:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346059136 00000100:00100000:2.0:1787826255.943838:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23094 00000100:00100000:2.0:1787826255.943846:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845affb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346059136:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826255.943906:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845affb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346059136:12345-0@lo:1000:mdt_io00_003.0 Request processed in 60us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826255.943913:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23094 00000100:00100000:1.0:1787826255.943982:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059136:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826255.944003:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x189:0x0] 00000100:00100000:1.0:1787826255.944099:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059264:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.944164:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.944176:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826255.944229:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346059264 00000100:00100000:2.0:1787826255.944234:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346059264 00000100:00100000:2.0:1787826255.944263:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23095 00000100:00100000:2.0:1787826255.944270:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346059264:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826255.944317:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346059264:12345-0@lo:1000:mdt_io00_003.0 Request processed in 47us (153us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826255.944324:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23095 00000100:00100000:1.0:1787826255.944392:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059264:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.944506:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059392:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826255.944535:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.944557:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.944763:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059392:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.944836:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059520:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826255.944863:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826255.944936:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346059520 00000100:00100000:2.0:1787826255.944941:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346059520 00000100:00100000:2.0:1787826255.944973:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23097 00000100:00100000:2.0:1787826255.944980:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346059520:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826255.945029:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:125380:x1874663346059520:12345-0@lo:1000:mdt_io00_003.0 Request processed in 49us (164us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826255.945047:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23097 00000100:00100000:0.0:1787826255.945098:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826255.945130:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0001-mdtlov_UUID:125380:1874663346059520:0@lo:1000:mdt_io00_003.0 00000040:00080000:0.0:1787826255.945617:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].222, 64 off22400 00000004:00080000:0.0:1787826255.945628:0:125380:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8068157522105ns, next at 8069157519688ns, rc = 0 00000040:00080000:0.0:1787826255.945670:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1162, 904 off1055496 00000040:00080000:0.0:1787826255.945722:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1480, 984 off1493552 00000004:00080000:0.0:1787826255.945729:0:125380:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78434e4c00 with version 6756 00000100:00100000:0.0:1787826255.946001:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346059776 00000100:00100000:0.0:1787826255.946008:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346059776 00000100:00100000:0.0:1787826255.946063:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23098 00000100:00100000:0.0:1787826255.946071:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346059776:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826255.946528:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346059776:12345-0@lo:1000:osp_up0-1.0 Request processed in 456us (646us total) trans 8589957273 rc 0/0 00000100:00100000:0.0:1787826255.946537:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23098 00000100:00100000:1.0:1787826255.947298:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346059776:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826255.947765:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947777:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947784:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947789:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826255.947794:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947801:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947816:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3b:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:0.0:1787826255.947819:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826255.947823:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947829:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48bf3 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826255.947864:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000403:0x1db1:0x0] 00010000:00010000:0.0:1787826255.947871:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48bec expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947879:0:125380:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48bec expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947889:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947894:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947899:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947904:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826255.947906:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947911:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947919:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f7843326200) 00010000:00010000:0.0:1787826255.947921:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826255.947924:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947929:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48b9f lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826255.947944:0:125380:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d162a5c7 reply state 000000002c9b4f60 transno 4294985292 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947955:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947962:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x5a734ce9 (ffff9f7845cb1c00) 00010000:00010000:0.0:1787826255.947965:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826255.947967:0:125380:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:0.0:1787826255.947971:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x189:0x0] 00010000:00010000:0.0:1787826255.947976:0:125380:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947984:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.947989:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.948017:0:125380:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826255.948021:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x189:0x0] 00010000:00010000:0.0:1787826255.948026:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.948878:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826255.948887:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.948901:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826255.948907:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826255.948914:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.948955:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.948962:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826255.948997:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.949005:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f48bc2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48bc9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826255.949023:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826255.949025:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826255.949026:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826255.949045:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826255.949064:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346059904:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826255.949154:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826255.949199:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346059904 00000100:00100000:1.0:1787826255.949206:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346059904 00000100:00100000:1.0:1787826255.949252:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10942 00000100:00100000:1.0:1787826255.949262:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b471050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346059904:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826255.949275:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826255.949292:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48bc9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48bc2 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949791:0:125380:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949806:0:125380:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949814:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949821:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949860:0:125380:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826255.949885:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949897:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949906:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949913:0:125380:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949919:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949925:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826255.949940:0:125380:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1787826255.949967:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:1.0:1787826255.949977:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826255.950017:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b471050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346059904:12345-0@lo:103:ldlm_bl.0 Request processed in 756us (863us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826255.950024:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10942 00000100:00100000:0.0:1787826255.950180:0:125380:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222916:x1874663332484096:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 68996us (69206us total) trans 4294985292 rc 0/0 00000100:00100000:0.0:1787826255.950188:0:125380:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1220 00010000:00010000:2.0:1787826255.953862:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826255.953877:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.953888:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826255.953895:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826255.953904:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.953951:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.953956:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826255.953996:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 1/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.954003:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f48bfa lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48c01 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.954026:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826255.954029:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.954044:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826255.954048:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.954055:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.954066:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3b:0x0].0x0 (ffff9f784b4a5d00) 00010000:00010000:2.0:1787826255.954069:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826255.954072:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.954077:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f48c08 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e3b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.954090:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826255.954092:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826255.954094:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826255.954979:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826255.955004:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346060032:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826255.955080:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826255.955203:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346060032 00000100:00100000:2.0:1787826255.955212:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346060032 00000100:00100000:2.0:1787826255.955269:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10943 00000100:00100000:2.0:1787826255.955279:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346060032:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826255.955293:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826255.955312:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48c01 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48bfa expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826255.955328:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f79462e8400) 00010000:00010000:2.0:1787826255.955338:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826255.955390:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346060032:12345-0@lo:103:ldlm_bl.0 Request processed in 112us (309us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826255.955398:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10943 00000100:00100000:2.0:1787826255.955438:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346060032:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826255.956260:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346059904:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826255.957445:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826256.044363:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826257.006923:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.006977:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332485376 00000100:00100000:2.0:1787826257.006990:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332485376 00000100:00100000:2.0:1787826257.007080:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68113 00000100:00100000:2.0:1787826257.007094:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:222952:x1874663332485376:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826257.007164:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826257.007195:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f48c0f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c68 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.007296:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f48c16 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.007316:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f48c16 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.007334:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f48c16 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826257.007485:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3a:0x0]: returning size 4096 00000004:00200000:2.0:1787826257.007492:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787826257.007518:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f48c16 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.007551:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f48c0f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c68 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.007579:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f48c16 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c68 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.007589:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:2.0:1787826257.007592:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826257.007597:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826257.007675:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222952:x1874663332485376:12345-192.168.203.56@tcp:101:rm.0 Request processed in 583us (765us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.007683:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68113 00000100:00080000:2.0:1787826257.007712:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826257.013724:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826257.013836:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332485504 00000100:00100000:0.0:1787826257.013855:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332485504 00000100:00100000:0.0:1787826257.013937:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68114 00000100:00100000:0.0:1787826257.013950:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222952:x1874663332485504:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826257.013981:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826257.014017:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f48c1d lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c6f expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.014133:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f48c24 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.014152:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f48c24 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.014166:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f48c24 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826257.014232:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18b:0x0]: returning size 4096 00000004:00200000:0.0:1787826257.014239:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826257.014285:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f48c1d lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c6f expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.014304:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f48c24 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c6f expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.014315:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:0.0:1787826257.014319:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826257.014324:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826257.014895:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222952:x1874663332485504:12345-192.168.203.56@tcp:101:rm.0 Request processed in 943us (1180us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.014906:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68114 00000100:00100000:2.0:1787826257.018618:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.018821:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332485632 00000100:00100000:1.0:1787826257.018835:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332485632 00000100:00100000:1.0:1787826257.018922:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68115 00000100:00100000:1.0:1787826257.018937:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:222952:x1874663332485632:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826257.018963:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.018996:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c2b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c76 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.019096:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f48c32 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.019143:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f48c32 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.019161:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f48c32 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826257.019225:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x189:0x0]: returning size 4096 00000004:00200000:1.0:1787826257.019231:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826257.019277:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c2b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x189:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c76 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.019308:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f48c32 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c76 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.019318:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x0 (ffff9f7967ac1100) 00010000:00010000:1.0:1787826257.019322:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826257.019326:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00080000:2.0:1787826257.019511:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826257.019587:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222952:x1874663332485632:12345-192.168.203.56@tcp:101:rm.0 Request processed in 653us (984us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.019596:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68115 00000100:00100000:2.0:1787826257.024471:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826257.025206:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332485888 00000100:00100000:0.0:1787826257.025221:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332485888 00000100:00100000:0.0:1787826257.025305:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68116 00000100:00100000:0.0:1787826257.025318:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222952:x1874663332485888:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826257.026310:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826257.026437:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f48c39 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c7d expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826257.026658:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3a:0x0]: returning size 4096 00010000:00010000:0.0:1787826257.026801:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f48c39 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c7d expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.026819:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:0.0:1787826257.026823:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826257.026829:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f48c39 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c7d expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.026849:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826257.026941:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:222952:x1874663332485888:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1624us (2485us total) trans 4294985293 rc 301/301 00000100:00100000:0.0:1787826257.026951:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68116 00000100:00100000:0.0:1787826257.043845:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.053384:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.065178:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000004:00080000:2.0:1787826257.066238:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6757 rpc_version 6757 00000020:00100000:2.0:1787826257.066280:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826257.066291:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d8bd500 x1874663346060288/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826257.066333:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bd500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346060288:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826257.066407:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.066440:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.066985:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bd500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346060288:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826257.069144:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826257.074962:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.078968:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332486784 00000100:00100000:1.0:1787826257.078983:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332486784 00000100:00100000:1.0:1787826257.079064:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68118 00000100:00100000:1.0:1787826257.079078:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222952:x1874663332486784:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:1.0:1787826257.079179:0:17813:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345561600 00000100:00100000:1.0:1787826257.079278:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222952:x1874663332486784:12345-192.168.203.56@tcp:41:rm.0 Request processed in 202us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.079286:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68118 00000100:00100000:2.0:1787826257.081326:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.081780:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332486912 00000100:00100000:1.0:1787826257.081794:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332486912 00000100:00100000:1.0:1787826257.081869:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68119 00000100:00100000:1.0:1787826257.081882:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:222952:x1874663332486912:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826257.081912:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.081940:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c5c lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c8b expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082039:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082056:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082071:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082085:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082093:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082101:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082220:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48c71 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082228:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48c71 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082236:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48c71 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826257.082261:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48c71 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082283:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082290:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082298:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082305:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082311:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826257.082315:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082323:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082337:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:1.0:1787826257.082341:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826257.082346:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082353:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f48c63 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082368:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082375:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082382:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082387:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826257.082390:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082396:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082404:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x5a734ce9 (ffff9f7845cb1c00) 00010000:00010000:1.0:1787826257.082406:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826257.082409:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082414:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f48c6a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082444:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c5c lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c8b expref: 16 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082459:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48c71 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c8b expref: 15 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.082467:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7967ac1300) 00010000:00010000:1.0:1787826257.082469:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826257.082472:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826257.082545:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:222952:x1874663332486912:12345-192.168.203.56@tcp:101:rm.0 Request processed in 666us (1232us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.082558:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68119 00000100:00100000:2.0:1787826257.088054:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.088164:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332487040 00000100:00100000:1.0:1787826257.088178:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332487040 00000100:00100000:1.0:1787826257.088255:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68120 00000100:00100000:1.0:1787826257.088268:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:222952:x1874663332487040:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826257.088294:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.088324:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48c78 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1db1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c92 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.088410:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48c7f lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.088427:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48c7f lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.088440:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48c7f lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826257.088498:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db1:0x0]: returning size 6 00000004:00200000:1.0:1787826257.088503:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:1.0:1787826257.088546:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48c78 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c92 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.088566:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48c7f lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c92 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.088576:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7843326400) 00010000:00010000:1.0:1787826257.088579:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826257.088586:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826257.088663:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222952:x1874663332487040:12345-192.168.203.56@tcp:101:rm.0 Request processed in 397us (624us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.088671:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68120 00000100:00100000:0.0:1787826257.093302:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.093410:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332487168 00000100:00100000:2.0:1787826257.093425:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332487168 00000100:00100000:2.0:1787826257.093508:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10944 00000100:00100000:2.0:1787826257.093521:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:2394:x1874663332487168:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826257.093540:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826257.093584:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48c71 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c8b expref: 15 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.093606:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7967ac1300) 00010000:00010000:2.0:1787826257.093617:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826257.093693:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:2394:x1874663332487168:12345-192.168.203.56@tcp:103:rm.0 Request processed in 174us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.093701:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10944 00000100:00100000:0.0:1787826257.103658:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332487424 00000100:00100000:0.0:1787826257.103671:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332487424 00000100:00100000:0.0:1787826257.103758:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68121 00000100:00100000:0.0:1787826257.103771:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222952:x1874663332487424:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826257.103846:0:35092:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826257.103882:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48c7f lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c92 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.103913:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7843326400) 00010000:00010000:0.0:1787826257.103924:0:35092:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826257.104033:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346060416:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826257.104102:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.104150:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826257.106154:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346060416:0@lo:1000:mdt00_009.0 00010000:00010000:0.0:1787826257.106273:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.106285:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.106310:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.106333:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346060544:0@lo:101:mdt00_009.0 00000100:00100000:0.0:1787826257.106402:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.106432:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78494b8d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.111693:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346060672:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826257.111750:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f48c8d lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48c86 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.111767:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:2.0:1787826257.111771:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826257.111777:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f48c8d lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48c86 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.111791:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:2.0:1787826257.111888:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346060544:12345-0@lo:101:mdt00_009.0 Request processed in 5142us (5483us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.111897:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23101 00000100:00100000:0.0:1787826257.112280:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a300 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346060544:0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826257.112378:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.112393:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.117062:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826257.118593:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346060800 00000100:00100000:0.0:1787826257.118599:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346060800 00000100:00100000:0.0:1787826257.118642:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7778 00000100:00100000:0.0:1787826257.118651:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6513:x1874663346060800:12345-0@lo:105: 00010000:00010000:0.0:1787826257.118663:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.118674:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.118684:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.118691:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.118699:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.118707:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.118740:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42a00) 00000100:00100000:0.0:1787826257.118790:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6513:x1874663346060800:12345-0@lo:105: Request processed in 141us (465us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.118798:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7778 00000100:00100000:0.0:1787826257.118851:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6513:1874663346060800:0@lo:105: 00010000:00010000:0.0:1787826257.118873:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826257.118928:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:2393:x1874663332487552:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 923us (1882us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.118937:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21796 00000100:00100000:1.0:1787826257.122821:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346060928 00000100:00100000:1.0:1787826257.122832:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346060928 00000100:00100000:1.0:1787826257.122907:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23102 00000100:00100000:1.0:1787826257.122918:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346060928:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826257.122939:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.122958:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48c94 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.122988:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 2/0,1 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.123000:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48c94 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.123019:0:6533:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:00080000:1.0:1787826257.123064:0:6533:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00010000:00010000:1.0:1787826257.123142:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48c94 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.123152:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x5a734ce9 (ffff9f7845cb1c00) 00010000:00010000:1.0:1787826257.123156:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826257.123160:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48c94 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.123174:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826257.123236:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:35092:x1874663346060928:12345-0@lo:101:mdt00_009.0 Request processed in 319us (503us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.123242:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23102 00010000:00010000:1.0:1787826257.124971:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826257.124995:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.125059:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.125067:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826257.125149:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346061056:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787826257.125234:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826257.126467:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f48bd7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x189:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48bde expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126498:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48bec expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826257.126508:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48bec expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126546:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48bec expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126553:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826257.126576:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f48be5 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48bec expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826257.126594:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985295 is committed 00000001:00080000:1.0:1787826257.126627:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985292 is committed 00000004:00080000:1.0:1787826257.126633:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 359 committed 00000001:00080000:1.0:1787826257.126648:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985290 is committed 00000020:00080000:1.0:1787826257.126666:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3377 batchid updated 00000100:00100000:1.0:1787826257.126700:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346061184:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826257.126759:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826257.126829:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126838:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126846:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126854:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826257.126859:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126867:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126880:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x5a734ce9 (ffff9f7845cb1c00) 00010000:00010000:1.0:1787826257.126883:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826257.126889:0:3638:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48c94 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126906:0:3638:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48c94 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126917:0:3638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48c94 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.126951:0:3638:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48c94 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.126963:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0001_UUID:3638:1874663346061312:0@lo:105: 00000100:00100000:1.0:1787826257.126996:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.127042:0:3638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358b40 going to sleep for 11 seconds 00000020:00080000:1.0:1787826257.127163:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3377 00000004:00080000:1.0:1787826257.127239:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784a358900 with version 6758 00000020:00080000:1.0:1787826257.127250:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3374 cancel update log [0x200000401:0x3:0x0].1476: rc = 0 00000020:00080000:1.0:1787826257.127345:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3374 cancel update log [0x240000400:0x3:0x0].1158: rc = 0 00000004:00080000:1.0:1787826257.127369:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784631ab40 with version 6759 00000020:00080000:1.0:1787826257.127496:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3375 cancel update log [0x200000401:0x3:0x0].1477: rc = 0 00000020:00080000:1.0:1787826257.127522:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3375 cancel update log [0x240000400:0x3:0x0].1159: rc = 0 00000004:00080000:1.0:1787826257.127545:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784631a780 with version 6760 00000020:00080000:1.0:1787826257.127548:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3376 cancel update log [0x200000401:0x3:0x0].1478: rc = 0 00000020:00080000:1.0:1787826257.127566:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3376 cancel update log [0x240000400:0x3:0x0].1160: rc = 0 00000004:00080000:1.0:1787826257.127587:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784adfc780 with version 6761 00000020:00080000:1.0:1787826257.127589:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3377 cancel update log [0x200000401:0x3:0x0].1479: rc = 0 00000020:00080000:1.0:1787826257.127608:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3377 cancel update log [0x240000400:0x3:0x0].1161: rc = 0 00000020:00080000:1.0:1787826257.127614:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3377 00000100:00100000:1.0:1787826257.127635:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346061312 00000100:00100000:1.0:1787826257.127645:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346061312 00000100:00100000:1.0:1787826257.127704:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7779 00000100:00100000:1.0:1787826257.127715:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:3638:x1874663346061312:12345-0@lo:105: 00010000:00010000:1.0:1787826257.127729:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.127741:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.127751:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.127761:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 5/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.127769:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 6/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.127779:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 6/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.127806:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69e00) 00000100:00100000:1.0:1787826257.127866:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:3638:x1874663346061312:12345-0@lo:105: Request processed in 152us (869us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.127879:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7779 00000100:00100000:1.0:1787826257.127912:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844295180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346061440:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826257.127948:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1787826257.127986:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6758 rpc_version 6758 00000020:00100000:1.0:1787826257.128009:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826257.128017:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f79785e6680 x1874663346061696/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826257.128044:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346061696:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.128074:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.128090:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.128150:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0001_UUID:3638:1874663346061312:0@lo:105: 00010000:00010000:1.0:1787826257.128177:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.128186:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48bbb lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 125380 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.128220:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346061440 00000100:00100000:1.0:1787826257.128225:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346061440 00000100:00100000:1.0:1787826257.128277:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30755 00000100:00100000:1.0:1787826257.128284:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346061440:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1787826257.128325:0:8445:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1224 count 1 00010000:00010000:1.0:1787826257.128356:0:8445:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.129487:0:8445:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 5 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.129509:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c8:0x0].0x0 since=1s v=(1 0) 00010000:00010000:1.0:1787826257.129514:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826257.129520:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.129634:0:8445:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 5 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.129654:0:8445:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 5 pid: 35097 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.129670:0:8445:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346061824:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826257.129794:0:8445:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78457853c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.129825:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346061696 00000100:00100000:1.0:1787826257.129833:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346061696 00000100:00100000:1.0:1787826257.129885:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23103 00000100:00100000:1.0:1787826257.129894:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346061696:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.130128:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346061696:12345-0@lo:1000:osp_up0-1.0 Request processed in 234us (2052us total) trans 8589957275 rc 0/0 00000100:00100000:1.0:1787826257.130138:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23103 00000100:00080000:2.0:1787826257.130664:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826257.130898:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346061696:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826257.130914:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6759 rpc_version 6759 00000020:00100000:1.0:1787826257.130938:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826257.130945:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f79785e5500 x1874663346062080/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826257.130967:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346062080:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.131014:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.131055:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.131073:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346062080 00000100:00100000:1.0:1787826257.131079:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346062080 00000100:00100000:1.0:1787826257.131145:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23104 00000100:00100000:1.0:1787826257.131155:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346062080:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.131285:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346062080:12345-0@lo:1000:osp_up0-1.0 Request processed in 130us (269us total) trans 8589957276 rc 0/0 00000100:00100000:1.0:1787826257.131291:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23104 00000100:00100000:1.0:1787826257.131326:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346061184 00000100:00100000:1.0:1787826257.131330:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346061184 00000100:00100000:1.0:1787826257.131367:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10946 00000100:00100000:1.0:1787826257.131374:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346061184:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826257.131386:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.131414:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f48bec lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48be5 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.131435:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7843326400) 00010000:00010000:1.0:1787826257.131448:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.131480:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346061184:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 108us (4722us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.131488:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10946 00000100:00100000:1.0:1787826257.131519:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346061184:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826257.131570:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346062080:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826257.131578:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6760 rpc_version 6760 00000020:00100000:1.0:1787826257.131591:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826257.131596:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc31500 x1874663346062336/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826257.131611:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346062336:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.131653:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.131670:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.131685:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346062336 00000100:00100000:1.0:1787826257.131688:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346062336 00000100:00100000:1.0:1787826257.131710:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23105 00000100:00100000:1.0:1787826257.131716:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346062336:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.131798:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346062336:12345-0@lo:1000:osp_up0-1.0 Request processed in 82us (144us total) trans 8589957277 rc 0/0 00000100:00100000:1.0:1787826257.131805:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23105 00000100:00100000:1.0:1787826257.131839:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346062336:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826257.131846:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6761 rpc_version 6761 00000020:00100000:1.0:1787826257.131857:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826257.131861:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7854624380 x1874663346062592/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826257.131873:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346062592:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.131894:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.131907:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.131917:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346062592 00000100:00100000:1.0:1787826257.131920:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346062592 00000100:00100000:1.0:1787826257.131937:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23106 00000100:00100000:1.0:1787826257.131942:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346062592:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.132009:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346062592:12345-0@lo:1000:osp_up0-1.0 Request processed in 65us (112us total) trans 8589957278 rc 0/0 00000100:00100000:1.0:1787826257.132015:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23106 00000100:00100000:1.0:1787826257.132058:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346062592:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826257.132082:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.132095:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826257.132157:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18b:0x0] 00000100:00100000:1.0:1787826257.132216:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346062720:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826257.132241:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.132256:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.132272:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346062720 00000100:00100000:1.0:1787826257.132275:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346062720 00000100:00100000:1.0:1787826257.132300:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23107 00000100:00100000:1.0:1787826257.132306:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346062720:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826257.132359:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346062720:12345-0@lo:1000:mdt00_009.0 Request processed in 53us (116us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.132366:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23107 00000100:00100000:1.0:1787826257.132395:0:35092:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985290 gen 2 00000004:00080000:1.0:1787826257.132401:0:35092:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985290 committed transno 4294985290 00000100:00100000:1.0:1787826257.132419:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346062720:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826257.132461:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346062848:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826257.132484:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.132497:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.132506:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346062848 00000100:00100000:1.0:1787826257.132509:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346062848 00000100:00100000:1.0:1787826257.132528:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23108 00000100:00100000:1.0:1787826257.132533:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346062848:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826257.132590:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346062848:12345-0@lo:1000:mdt00_009.0 Request processed in 57us (104us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.132596:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23108 00000100:00100000:1.0:1787826257.132617:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346062848:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826257.132652:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346062976:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826257.132672:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.132684:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.132694:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346062976 00000100:00100000:1.0:1787826257.132697:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346062976 00000100:00100000:1.0:1787826257.132716:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23109 00000100:00100000:1.0:1787826257.132722:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346062976:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826257.132796:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346062976:12345-0@lo:1000:mdt00_009.0 Request processed in 73us (123us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826257.132802:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23109 00000100:00100000:1.0:1787826257.132827:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346062976:0@lo:1000:mdt00_009.0 00010000:00010000:1.0:1787826257.132883:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.132892:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.132906:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.132917:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346063104:0@lo:101:mdt00_009.0 00000100:00100000:1.0:1787826257.132939:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.132952:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.132961:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346063104 00000100:00100000:1.0:1787826257.132964:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346063104 00000100:00100000:1.0:1787826257.132982:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23110 00000100:00100000:1.0:1787826257.132988:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346063104:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826257.132998:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.133015:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48cb0 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48ca9 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.133051:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48cb0 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48ca9 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.133065:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48cb0 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48ca9 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.133074:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:1.0:1787826257.133077:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826257.133081:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826257.133135:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:35092:x1874663346063104:12345-0@lo:101:mdt00_009.0 Request processed in 146us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.133141:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23110 00000100:00100000:1.0:1787826257.133167:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346063104:0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826257.133180:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.133190:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.133213:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.133220:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.133228:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787826257.133583:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].208, 64 off21504 00000004:00080000:1.0:1787826257.133597:0:35092:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8069345490620ns, next at 8070345487676ns, rc = 0 00000040:00080000:1.0:1787826257.133639:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].926, 384 off871384 00000040:00080000:1.0:1787826257.133684:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].929, 464 off969808 00000004:00080000:1.0:1787826257.133690:0:35092:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7845785a80 with version 1807 00000100:00100000:2.0:1787826257.139961:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346061056:0@lo:103:jbd2/dm-1-8.0 00000004:00080000:2.0:1787826257.140010:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1807 rpc_version 1807 00000020:00100000:2.0:1787826257.140065:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826257.140074:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc31f80 x1874663346063360/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1664/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826257.140101:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346063360:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826257.140447:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.140490:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.140523:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346063360 00000100:00100000:2.0:1787826257.140530:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346063360 00000100:00100000:2.0:1787826257.140732:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23111 00000100:00100000:2.0:1787826257.140743:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346063360:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826257.141191:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346063360:12345-0@lo:1000:osp_up1-0.0 Request processed in 449us (745us total) trans 4294985296 rc 0/0 00000100:00100000:2.0:1787826257.141200:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23111 00000100:00100000:2.0:1787826257.141260:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346063360:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787826257.143245:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.143278:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.143364:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144441:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144454:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144461:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144484:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787826257.144520:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144526:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144533:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144610:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826257.144617:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144627:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144647:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:2.0:1787826257.144651:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826257.144657:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144665:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48cb7 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144685:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144694:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144702:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144723:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826257.144742:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18b:0x0] 00010000:00010000:2.0:1787826257.144752:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144759:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144766:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144778:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:2.0:1787826257.144783:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18b:0x0] 00010000:00010000:2.0:1787826257.144788:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144797:0:35092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f48c94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48c9b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.144914:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:222952:x1874663332487424:12345-192.168.203.56@tcp:36:rm.0 Request processed in 41139us (41375us total) trans 8589957279 rc 0/0 00000100:00100000:2.0:1787826257.144924:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68121 00010000:00010000:2.0:1787826257.144982:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1787826257.144989:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.144996:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826257.144999:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145006:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145016:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:2.0:1787826257.145018:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826257.145022:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145042:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48cbe lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145059:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826257.145063:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145069:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826257.145073:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826257.145082:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145146:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145154:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826257.145178:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 1/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145184:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f48ca9 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48cb0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145198:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826257.145201:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145207:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826257.145211:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826257.145218:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145237:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145243:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826257.145248:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145255:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42a00/0xe21309aa81f48c86 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48c8d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145266:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826257.145268:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826257.145269:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826257.145276:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826257.145304:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346063616:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826257.145377:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.145433:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346063616 00000100:00100000:2.0:1787826257.145443:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346063616 00000100:00100000:2.0:1787826257.145499:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10947 00000100:00100000:2.0:1787826257.145510:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346063616:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826257.145525:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826257.145558:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f48c8d lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48c86 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.145574:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:2.0:1787826257.145581:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826257.145608:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346063616:12345-0@lo:103:ldlm_bl.0 Request processed in 101us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.145616:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10947 00000100:00100000:2.0:1787826257.145664:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346063616:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826257.145682:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826257.145688:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346063488:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826257.145715:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.146326:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:1.0:1787826257.146349:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826257.146367:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346063488 00000100:00100000:1.0:1787826257.146373:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346063488 00000100:00100000:1.0:1787826257.146411:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332487680 00000100:00100000:1.0:1787826257.146414:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332487680 00010000:00010000:1.0:1787826257.146442:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 6 pid: 35097 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.146467:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21797 00000100:00100000:1.0:1787826257.146476:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332487680:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.146487:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.146492:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 6 pid: 35097 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.146506:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 6 pid: 35097 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.146527:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826257.146532:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826257.146540:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.146985:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.147012:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.147072:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332487680:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 595us (749us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.147080:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21797 00000100:00100000:1.0:1787826257.147102:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10948 00000100:00100000:1.0:1787826257.147134:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b470450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346063488:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.147144:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.147160:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48cb0 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48ca9 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.147184:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db1:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:1.0:1787826257.147192:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.147229:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b470450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346063488:12345-0@lo:103:ldlm_bl.0 Request processed in 96us (1514us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.147235:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10948 00000100:00100000:1.0:1787826257.150834:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.150891:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:2.0:1787826257.150973:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826257.150999:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332487808 00000100:00100000:2.0:1787826257.151011:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332487808 00000100:00100000:2.0:1787826257.151129:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10949 00000100:00100000:2.0:1787826257.151143:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332487808:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826257.151165:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826257.151197:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48ae9 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c14 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.151221:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f7843326e00) 00010000:00010000:2.0:1787826257.151227:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826257.151305:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332487808:12345-192.168.203.56@tcp:103:rm.0 Request processed in 165us (412us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.151313:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10949 00010000:00010000:2.0:1787826257.151388:0:8445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f48bd0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c61 expref: 5 pid: 35097 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.151404:0:8445:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346061824:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826257.151443:0:8445:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.151462:0:8445:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.151470:0:8445:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.151481:0:8445:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.151489:0:8445:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.151507:0:8445:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826257.151915:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3655:x1874663346061440:12345-0@lo:6:osp-syn-0-1.0 Request processed in 23629us (23963us total) trans 4294973830 rc 0/0 00000100:00100000:2.0:1787826257.151926:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30755 00010000:00010000:2.0:1787826257.151958:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826257.151965:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.151974:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826257.151980:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.151990:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.152009:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.152017:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000430ce35c/0xe21309aa81f48ca2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.152050:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826257.152052:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826257.152054:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1787826257.152703:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7844295180/2, rc 0, transno 4294973830 00000100:00100000:2.0:1787826257.152713:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844295180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346061440:0@lo:6:osp-syn-0-1.0 00000100:00100000:2.0:1787826257.152924:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346063488:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826257.164694:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826257.164729:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.165284:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332488448 00000100:00100000:1.0:1787826257.165293:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332488448 00010000:00010000:1.0:1787826257.165354:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c37 expref: 8 pid: 35092 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.165379:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332488576 00000100:00100000:1.0:1787826257.165382:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332488576 00010000:00010000:1.0:1787826257.165395:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c0d expref: 9 pid: 6529 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.165415:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21798 00000100:00100000:1.0:1787826257.165425:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332488448:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.165443:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.165451:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c37 expref: 8 pid: 35092 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165464:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c37 expref: 8 pid: 35092 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165497:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48b36 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c37 expref: 8 pid: 35092 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165518:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826257.165523:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48ccc lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48cc5 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165532:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.165582:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332488448:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 158us (904us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.165590:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21798 00000100:00100000:1.0:1787826257.165607:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21799 00000100:00100000:1.0:1787826257.165614:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332488576:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.165622:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.165628:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c0d expref: 9 pid: 6529 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165639:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c0d expref: 9 pid: 6529 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165653:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48acd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c0d expref: 9 pid: 6529 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165664:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826257.165669:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48ccc lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48cc5 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165681:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48ccc lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48cc5 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165691:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48ccc lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48cc5 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165730:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48ccc lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48cc5 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.165747:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346064128:0@lo:105: 00000100:00100000:1.0:1787826257.165796:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.165841:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78474d7780 going to sleep for 11 seconds 00000100:00080000:1.0:1787826257.165865:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826257.165885:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346064128 00000100:00100000:1.0:1787826257.165889:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346064128 00000100:00100000:1.0:1787826257.165918:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7780 00000100:00100000:1.0:1787826257.165925:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346064128:12345-0@lo:105: 00010000:00010000:1.0:1787826257.165937:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165948:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165958:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165966:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165974:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165984:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.165999:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68a00) 00000100:00100000:1.0:1787826257.166043:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346064128:12345-0@lo:105: Request processed in 119us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.166050:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7780 00000100:00100000:1.0:1787826257.166085:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346064128:0@lo:105: 00010000:00010000:1.0:1787826257.166130:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.166151:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332488576:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 538us (1422us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.166158:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21799 00000100:00100000:0.0:1787826257.166382:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346063744:0@lo:101:mdt00_008.0 00010000:00010000:0.0:1787826257.166402:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826257.166425:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826257.166471:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.166479:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.166489:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.166506:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346064256:0@lo:101:mdt00_008.0 00000100:00100000:0.0:1787826257.166545:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.166577:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78494b8d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.171223:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346064256 00000100:00100000:1.0:1787826257.171249:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346064256 00000100:00100000:1.0:1787826257.171336:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23113 00000100:00100000:1.0:1787826257.171350:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35080:x1874663346064256:12345-0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826257.171380:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.171410:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f48cda lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48cd3 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.171451:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f48cda lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48cd3 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.171474:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f48cda lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48cd3 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.171484:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1300) 00010000:00010000:1.0:1787826257.171487:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826257.171492:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826257.171601:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:35080:x1874663346064256:12345-0@lo:101:mdt00_008.0 Request processed in 254us (5054us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.171608:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23113 00000100:00100000:0.0:1787826257.171972:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346064256:0@lo:101:mdt00_008.0 00010000:00010000:0.0:1787826257.171998:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.172011:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826257.172059:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826257.172155:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346064384:0@lo:1000:mdt00_008.0 00000100:00100000:0.0:1787826257.172209:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.172243:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826257.172577:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346064384:0@lo:1000:mdt00_008.0 00000100:00100000:0.0:1787826257.172625:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346064512:0@lo:1000:mdt00_008.0 00000100:00100000:0.0:1787826257.172652:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.172677:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.172773:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346064512 00000100:00100000:1.0:1787826257.172777:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346064512 00000100:00100000:1.0:1787826257.172811:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23115 00000100:00100000:1.0:1787826257.172818:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:35080:x1874663346064512:12345-0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826257.172940:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:35080:x1874663346064512:12345-0@lo:1000:mdt00_008.0 Request processed in 123us (287us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826257.172947:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23115 00000100:00100000:0.0:1787826257.173012:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346064512:0@lo:1000:mdt00_008.0 00010000:00010000:0.0:1787826257.173085:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.173095:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.173131:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.173145:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346064640:0@lo:101:mdt00_008.0 00000100:00100000:0.0:1787826257.173170:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.173188:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.173238:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346064640 00000100:00100000:1.0:1787826257.173243:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346064640 00000100:00100000:1.0:1787826257.173277:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23116 00000100:00100000:1.0:1787826257.173284:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:35080:x1874663346064640:12345-0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826257.173303:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.173320:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48ce8 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48ce1 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.173347:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c3e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.173363:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48ce8 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48ce1 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.173402:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c3e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.173418:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c3e expref: 6 pid: 6529 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.173429:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346064768:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826257.173511:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f753900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826257.176358:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:2.0:1787826257.176705:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f48b52 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c3e expref: 7 pid: 6529 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.176721:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346064768:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826257.176755:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48ce8 lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48ce1 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.176762:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f7843326e00) 00010000:00010000:2.0:1787826257.176766:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826257.176768:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:2.0:1787826257.176816:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+16:35080:x1874663346064640:12345-0@lo:101:mdt00_008.0 Request processed in 3532us (3645us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.176828:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23116 00000100:00100000:0.0:1787826257.177148:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346064896 00000100:00100000:0.0:1787826257.177154:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346064896 00000100:00100000:0.0:1787826257.177190:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7781 00000100:00100000:0.0:1787826257.177199:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346064896:12345-0@lo:105: 00010000:00010000:0.0:1787826257.177215:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177227:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177237:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177246:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177252:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177262:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177278:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43000) 00000100:00100000:0.0:1787826257.177606:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346064896:12345-0@lo:105: Request processed in 345us (774us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.177614:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7781 00000100:00100000:0.0:1787826257.177660:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346064640:0@lo:101:mdt00_008.0 00010000:00010000:0.0:1787826257.177682:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177721:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177733:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177742:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177765:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177772:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177778:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177806:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177812:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.177820:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.177836:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346065024:0@lo:101:mdt00_008.0 00000100:00100000:0.0:1787826257.177867:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.177929:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.183237:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346065024 00000100:00100000:1.0:1787826257.183380:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346065024 00000100:00100000:1.0:1787826257.183628:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23117 00000100:00100000:1.0:1787826257.183644:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+16:35080:x1874663346065024:12345-0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826257.183674:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.183704:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d04 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48cfd expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.183808:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f48c32 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c76 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.183821:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d04 lrc: 4/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48cfd expref: 17 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.183868:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f48c32 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c76 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.183886:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f48c32 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x189:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c76 expref: 7 pid: 35080 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.183904:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346065152:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826257.184035:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f753900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.188433:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346065152:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826257.188473:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d04 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48cfd expref: 17 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.188485:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x0 (ffff9f7967ac1100) 00010000:00010000:1.0:1787826257.188487:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826257.188492:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d04 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48cfd expref: 17 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.188503:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d04 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48cfd expref: 17 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.188511:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826257.188590:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:35080:x1874663346065024:12345-0@lo:101:mdt00_008.0 Request processed in 4952us (10723us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.188598:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23117 00000100:00100000:0.0:1787826257.191652:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346065024:0@lo:101:mdt00_008.0 00010000:00010000:0.0:1787826257.191690:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.192540:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346065408:0@lo:403:mdt00_008.0 00000100:00100000:0.0:1787826257.192851:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.192937:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.193018:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346065408 00000100:00100000:1.0:1787826257.193038:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346065408 00000100:00100000:1.0:1787826257.193121:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23118 00000100:00100000:1.0:1787826257.193132:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:35080:x1874663346065408:12345-0@lo:403:mdt00_008.0 00000100:00100000:1.0:1787826257.196924:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:35080:x1874663346065408:12345-0@lo:403:mdt00_008.0 Request processed in 3782us (4061us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.196942:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23118 00000100:00100000:0.0:1787826257.197098:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346065408:0@lo:403:mdt00_008.0 00000040:00080000:0.0:1787826257.201434:0:35080:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1164, 952 off1056792 00000040:00080000:0.0:1787826257.201863:0:35080:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1482, 1032 off1495008 00000004:00080000:0.0:1787826257.201873:0:35080:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78434e4f00 with version 6762 00000004:00080000:1.0:1787826257.202055:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6762 rpc_version 6762 00000020:00100000:1.0:1787826257.202149:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826257.202166:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f797c322d80 x1874663346065664/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826257.202203:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346065664:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.202972:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.203036:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f753900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.203132:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346065664 00000100:00100000:2.0:1787826257.203143:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346065664 00000100:00100000:2.0:1787826257.203217:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23119 00000100:00100000:2.0:1787826257.203229:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:7471:x1874663346065664:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826257.203970:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+18:7471:x1874663346065664:12345-0@lo:1000:osp_up0-1.0 Request processed in 739us (997us total) trans 8589957280 rc 0/0 00000100:00100000:2.0:1787826257.203981:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23119 00000100:00100000:1.0:1787826257.204181:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346065664:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787826257.204276:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3a:0x0]: returning size 4096 00010000:00010000:2.0:1787826257.204374:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204387:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204396:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204404:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826257.204409:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204418:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204439:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:2.0:1787826257.204442:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826257.204448:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204459:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f48cf6 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204487:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204496:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204504:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204617:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826257.204629:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x189:0x0] 00010000:00010000:2.0:1787826257.204637:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204644:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204649:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204654:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826257.204657:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204664:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204678:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:2.0:1787826257.204680:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826257.204684:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204690:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f48cef lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204700:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204708:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.204715:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.204844:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826257.204851:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.204859:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826257.204864:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826257.204872:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.204914:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 2/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.204921:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826257.204945:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 1/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.204952:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f48cfd lrc: 0/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48d04 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.204967:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826257.204970:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.204976:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826257.204980:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826257.204987:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.205001:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.205006:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826257.205012:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.205018:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826257.205020:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826257.205022:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826257.205052:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346065792:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826257.205134:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.205172:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826257.205178:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346065920:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826257.205201:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826257.205397:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:2.0:1787826257.205403:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f48ce1 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48ce8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826257.205432:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826257.205440:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826257.205448:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.205457:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.205468:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.205482:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826257.205485:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826257.205491:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.205498:0:35080:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.205710:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826257.205716:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.205725:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826257.205730:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.205851:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:222952:x1874663332488320:12345-192.168.203.56@tcp:36:rm.0 Request processed in 48741us (48915us total) trans 4294985298 rc 0/0 00000100:00100000:2.0:1787826257.205862:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68122 00000004:00010000:1.0:1787826257.205924:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.205953:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.205958:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826257.206008:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346066048:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826257.206057:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826257.206126:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.206135:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48cc5 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48ccc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.206148:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826257.206150:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826257.206152:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826257.206411:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346065792 00000100:00100000:1.0:1787826257.206419:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346065792 00000100:00100000:1.0:1787826257.206468:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346065920 00000100:00100000:1.0:1787826257.206471:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346065920 00000100:00100000:1.0:1787826257.206486:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346066048 00000100:00100000:1.0:1787826257.206489:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346066048 00000100:00100000:1.0:1787826257.206509:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10950 00000100:00100000:1.0:1787826257.206518:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943ed7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+20:3654:x1874663346065792:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.206534:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.206569:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d04 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x189:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48cfd expref: 20 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.206587:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x189:0x0].0x0 (ffff9f7967ac1100) 00010000:00010000:1.0:1787826257.206598:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.206637:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943ed7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+19:3654:x1874663346065792:12345-0@lo:103:ldlm_bl.0 Request processed in 120us (1503us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.206645:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10950 00000100:00100000:1.0:1787826257.206696:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10951 00000100:00100000:1.0:1787826257.206702:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+18:3654:x1874663346065920:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.206710:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.206722:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48ce8 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48ce1 expref: 18 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.206733:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3a:0x0].0x0 (ffff9f7843326e00) 00010000:00010000:1.0:1787826257.206871:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.206898:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+17:3654:x1874663346065920:12345-0@lo:103:ldlm_bl.0 Request processed in 195us (1696us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.206904:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10951 00000100:00100000:1.0:1787826257.206918:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10952 00000100:00100000:1.0:1787826257.206924:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+16:3652:x1874663346066048:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.206931:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.206944:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48ccc lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48cc5 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.206956:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826257.206959:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.225352:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346066048:12345-0@lo:103:ldlm_bl.0 Request processed in 18410us (19276us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.225387:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10952 00000100:00100000:2.0:1787826257.225910:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346066048:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826257.229999:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346065920:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826257.230018:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346065792:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826257.289426:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332488960 00000100:00100000:2.0:1787826257.289441:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332488960 00000100:00100000:2.0:1787826257.289515:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68123 00000100:00100000:2.0:1787826257.289527:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222960:x1874663332488960:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826257.289581:0:6528:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345561600 00000100:00100000:2.0:1787826257.289673:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222960:x1874663332488960:12345-192.168.203.56@tcp:41:stat.0 Request processed in 148us (749us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.289682:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68123 00000100:00100000:1.0:1787826257.335287:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.335664:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332489088 00000100:00100000:2.0:1787826257.335678:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332489088 00000100:00100000:2.0:1787826257.335762:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68124 00000100:00100000:2.0:1787826257.335775:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222964:x1874663332489088:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826257.335805:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826257.335832:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48d0b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800c99 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.335949:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.335967:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.335980:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826257.336043:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826257.336048:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826257.336073:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.336102:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48d0b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.336158:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.336169:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826257.336172:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826257.336177:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826257.336262:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222964:x1874663332489088:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 489us (989us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.336270:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68124 00000100:00100000:2.0:1787826257.340239:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.340496:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332489216 00000100:00100000:1.0:1787826257.340510:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332489216 00000100:00100000:1.0:1787826257.340593:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68125 00000100:00100000:1.0:1787826257.340606:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222964:x1874663332489216:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826257.340633:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826257.340660:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d19 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ca0 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.340741:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.340758:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.340772:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.340791:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.340801:0:17813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f48cda lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48cd3 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.340810:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.340851:0:17813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f48cda lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f48cd3 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.340871:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850359500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346066176:0@lo:104: 00000100:00100000:1.0:1787826257.340940:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.340992:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f753900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826257.341150:0:6509:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826257.341165:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346066176 00000100:00100000:0.0:1787826257.341170:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346066176 00000100:00100000:0.0:1787826257.341204:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7783 00000100:00100000:0.0:1787826257.341213:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843321450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17813:x1874663346066176:12345-0@lo:104: 00010000:00010000:0.0:1787826257.341228:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.341294:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826257.341310:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843321450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17813:x1874663346066176:12345-0@lo:104: Request processed in 99us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.341318:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7783 00010000:00010000:2.0:1787826257.341578:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826257.341588:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.341599:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826257.341606:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826257.341616:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.341653:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.341661:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826257.341701:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.341710:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f48cd3 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48cda expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.341736:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826257.341738:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826257.341739:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826257.341914:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346066304 00000100:00100000:2.0:1787826257.341920:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346066304 00010000:00010000:2.0:1787826257.341946:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f48cda lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f48cd3 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.341978:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21802 00000100:00100000:2.0:1787826257.341986:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346066304:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826257.341997:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826257.345016:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850359500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346066176:0@lo:104: 00010000:00010000:1.0:1787826257.345070:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.345591:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346066432:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826257.345673:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.345844:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346066432 00000100:00100000:2.0:1787826257.345853:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346066432 00000100:00100000:2.0:1787826257.345913:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10953 00000100:00100000:2.0:1787826257.345925:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850359500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346066432:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826257.345943:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826257.345968:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48c9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48c94 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.345987:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18b:0x0].0x5a734ce9 (ffff9f7845cb1c00) 00010000:00010000:2.0:1787826257.345998:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826257.346062:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850359500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346066432:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 138us (388us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.346070:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10953 00000100:00100000:2.0:1787826257.346139:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346066432:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826257.346965:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346066560:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787826257.347020:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.347454:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346066560 00000100:00100000:1.0:1787826257.347464:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346066560 00000100:00100000:1.0:1787826257.347582:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30756 00000100:00100000:1.0:1787826257.347594:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346066560:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1787826257.347629:0:24218:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1307 count 1 00010000:00010000:1.0:1787826257.347659:0:24218:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.347686:0:24218:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f48ab8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c06 expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.347696:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x51b:0x0].0x0 since=6s v=(0 0) 00010000:00010000:1.0:1787826257.347701:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826257.347706:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.347740:0:24218:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f48ab8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c06 expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.347756:0:24218:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f48ab8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c06 expref: 5 pid: 24218 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.347768:0:24218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:24218:1874663346066688:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826257.347851:0:24218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f753900 going to sleep for 11 seconds 00010000:00010000:1.0:1787826257.350771:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350784:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350872:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350881:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350890:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350898:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350905:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826257.350910:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350917:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350934:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826257.350937:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826257.350943:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350949:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f48d20 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350965:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350970:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350975:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350979:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826257.350982:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350987:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.350994:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1300) 00010000:00010000:1.0:1787826257.350996:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826257.350999:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.351004:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f48d27 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.351071:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d19 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca0 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.351082:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826257.351084:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826257.351088:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48d19 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca0 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.351097:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826257.351212:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:222964:x1874663332489216:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 10608us (10987us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826257.351222:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68125 00000100:00100000:2.0:1787826257.355506:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332489472 00000100:00100000:2.0:1787826257.355512:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332489472 00000100:00100000:2.0:1787826257.355587:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68126 00000100:00100000:2.0:1787826257.355600:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:222964:x1874663332489472:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826257.355634:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826257.355661:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48d35 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ca7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.355742:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.355759:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.355772:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826257.355828:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826257.355832:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826257.355856:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.355882:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48d35 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.355902:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.355912:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826257.355916:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826257.355919:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826257.355988:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:222964:x1874663332489472:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 390us (841us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.355999:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68126 00000100:00080000:0.0:1787826257.356211:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826257.356366:0:24218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:24218:1874663346066688:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826257.356414:0:24218:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.358848:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f48ab8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c06 expref: 6 pid: 24218 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.358913:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21803 00000100:00100000:2.0:1787826257.358929:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332489344:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826257.358948:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826257.358956:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f48ab8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c06 expref: 6 pid: 24218 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.358971:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f48ab8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c06 expref: 6 pid: 24218 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.358999:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f48ab8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c06 expref: 6 pid: 24218 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.359018:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826257.359035:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826257.359042:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.359054:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.359094:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826257.359200:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332489344:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 277us (4137us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.359209:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21803 00010000:00010000:1.0:1787826257.360220:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826257.360230:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.360238:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826257.360243:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.360252:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.360269:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.360276:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006782fc13/0xe21309aa81f48d2e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.360313:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826257.360315:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826257.360317:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1787826257.361394:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784d009c00/2, rc 0, transno 4294976139 00000100:00100000:2.0:1787826257.361409:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346066560:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787826257.361969:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332489728 00000100:00100000:2.0:1787826257.361979:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332489728 00000100:00080000:0.0:1787826257.362070:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826257.362761:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68127 00000100:00100000:2.0:1787826257.362777:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222964:x1874663332489728:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:2.0:1787826257.362933:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346066816:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826257.363005:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.363061:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.363630:0:6527:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957280 gen 2 00000004:00080000:2.0:1787826257.363637:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957273 committed transno 8589957280 00000020:00080000:2.0:1787826257.363644:0:6527:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78434e4000 refcount 2 committed 0 result 0 batchid 3378 00000020:00080000:2.0:1787826257.363648:0:6527:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629df00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78434e4c00 00000020:00080000:2.0:1787826257.363652:0:6527:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1480 00000020:00080000:2.0:1787826257.363654:0:6527:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784b76a400 00000020:00080000:2.0:1787826257.363657:0:6527:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1162 00000004:00080000:2.0:1787826257.363692:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957274 committed transno 8589957280 00000020:00080000:2.0:1787826257.363697:0:6527:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784548a780 refcount 2 committed 0 result 0 batchid 3379 00000020:00080000:2.0:1787826257.363700:0:6527:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2500 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784548a000 00000020:00080000:2.0:1787826257.363703:0:6527:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1481 00000020:00080000:2.0:1787826257.363705:0:6527:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784da79a00 00000020:00080000:2.0:1787826257.363707:0:6527:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1163 00000004:00080000:2.0:1787826257.363723:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957275 committed transno 8589957280 00000004:00080000:2.0:1787826257.363737:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957276 committed transno 8589957280 00000004:00080000:2.0:1787826257.363750:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957277 committed transno 8589957280 00000004:00080000:2.0:1787826257.363765:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957278 committed transno 8589957280 00000004:00080000:2.0:1787826257.363773:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957280 committed transno 8589957280 00000100:00100000:2.0:1787826257.363790:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346066816:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826257.363851:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5b100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346066944:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826257.363881:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.363902:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.363923:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346066944 00000100:00100000:2.0:1787826257.363928:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346066944 00000100:00100000:2.0:1787826257.363965:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23121 00000100:00100000:2.0:1787826257.363972:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6527:x1874663346066944:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826257.364092:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6527:x1874663346066944:12345-0@lo:1000:mdt00_000.0 Request processed in 120us (209us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826257.364099:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23121 00000100:00100000:2.0:1787826257.364159:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5b100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346066944:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787826257.364221:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f48d43 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.364233:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f48d43 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.364246:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f48d43 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.364257:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5b100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346067072:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787826257.364282:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.364309:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f56c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.364324:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346067072 00000100:00100000:2.0:1787826257.364328:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346067072 00000100:00100000:2.0:1787826257.364355:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23122 00000100:00100000:2.0:1787826257.364361:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6527:x1874663346067072:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787826257.364645:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826257.364665:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48d43 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.364695:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.364705:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.364714:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48d43 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.364744:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.364759:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 6 pid: 35095 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.364771:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5aa00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346067200:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826257.364868:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.364883:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 6 pid: 24702 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.364893:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346067328:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826257.364918:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826257.368658:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5aa00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346067200:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826257.368679:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346067328:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826257.368730:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48d43 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.368753:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826257.368756:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826257.368761:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48d43 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.368773:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:0.0:1787826257.369728:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346067072:12345-0@lo:101:mdt00_000.0 Request processed in 5357us (5436us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.369742:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23122 00000100:00100000:2.0:1787826257.370034:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5b100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346067072:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787826257.370063:0:6527:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f48d43 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48d4a expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.370076:0:6527:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f48d43 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48d4a expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.370208:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332489984 00000100:00100000:1.0:1787826257.370218:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332489984 00010000:00010000:1.0:1787826257.370261:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 7 pid: 24702 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.370331:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332489856 00000100:00100000:1.0:1787826257.370335:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332489856 00010000:00010000:1.0:1787826257.370350:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 7 pid: 35095 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.370391:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21804 00000100:00100000:1.0:1787826257.370401:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332489984:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.370423:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.370432:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 7 pid: 24702 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.370445:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 7 pid: 24702 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.370472:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48d12 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800c99 expref: 7 pid: 24702 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.370493:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826257.370500:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48d43 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.370510:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826257.370740:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21805 00000100:00100000:2.0:1787826257.370750:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7947dfcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332489856:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826257.370760:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826257.370766:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 7 pid: 35095 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.370777:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 7 pid: 35095 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.370791:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f48d3c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ca7 expref: 7 pid: 35095 timeout: 8169 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.370806:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826257.370811:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48d43 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.370822:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48d43 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.370830:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48d43 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.370870:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f48d4a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48d43 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.370886:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346067456:0@lo:105: 00000100:00100000:1.0:1787826257.370962:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332489984:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 561us (1112us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.370969:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21804 00000100:00100000:2.0:1787826257.372488:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.373678:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f56c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.373709:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346067456:0@lo:105: 00010000:00010000:2.0:1787826257.373729:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826257.373771:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dfcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332489856:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3021us (3871us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.373780:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21805 00000100:00100000:0.0:1787826257.374179:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346067712 00000100:00100000:0.0:1787826257.374186:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346067712 00000100:00100000:0.0:1787826257.374224:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23124 00000100:00100000:0.0:1787826257.374233:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346067712:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826257.374327:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346067712:12345-0@lo:1000:mdt00_000.0 Request processed in 95us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.374333:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23124 00000100:00100000:2.0:1787826257.374648:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346067840 00000100:00100000:2.0:1787826257.374654:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346067840 00000100:00100000:2.0:1787826257.374681:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23125 00000100:00100000:2.0:1787826257.374687:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346067840:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826257.378554:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346067840:12345-0@lo:1000:mdt00_000.0 Request processed in 3843us (4011us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826257.378574:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23125 00000100:00100000:0.0:1787826257.380206:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346067968 00000100:00100000:0.0:1787826257.380223:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346067968 00000100:00100000:0.0:1787826257.380416:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23126 00000100:00100000:0.0:1787826257.380429:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346067968:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826257.380640:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346067968:12345-0@lo:1000:mdt00_000.0 Request processed in 213us (1663us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826257.380649:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23126 00000100:00100000:1.0:1787826257.380726:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346067968:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826257.380827:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346068096:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826257.380869:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.381323:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78430d9000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.381725:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346068096:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826257.381789:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346068224:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826257.381815:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.381836:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78513b3d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.382089:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346068224:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826257.382155:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346068352:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826257.382178:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826257.382197:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78513b3d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.382209:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346068352 00000100:00100000:1.0:1787826257.382212:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346068352 00000100:00100000:1.0:1787826257.382233:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23129 00000100:00100000:1.0:1787826257.382239:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346068352:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826257.382282:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346068352:12345-0@lo:1000:mdt00_000.0 Request processed in 43us (103us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.382288:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23129 00000100:00100000:1.0:1787826257.382424:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346068352:0@lo:1000:mdt00_000.0 00000040:00080000:1.0:1787826257.383479:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1165, 2168 off1058960 00000040:00080000:1.0:1787826257.383622:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1483, 2368 off1497376 00000004:00080000:1.0:1787826257.383628:0:6527:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78513b3000 with version 6763 00000100:00100000:1.0:1787826257.384097:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346068608 00000100:00100000:1.0:1787826257.384249:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346068608 00000100:00100000:1.0:1787826257.384407:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23130 00000100:00100000:1.0:1787826257.384419:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346068608:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826257.385701:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346068608:12345-0@lo:1000:osp_up0-1.0 Request processed in 1278us (1719us total) trans 8589957281 rc 0/0 00000100:00100000:1.0:1787826257.385718:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23130 00000100:00100000:2.0:1787826257.385851:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346068608:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826257.386067:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.386096:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.386137:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.386163:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.386171:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.386178:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.386222:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.386230:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.386242:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.386266:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346068736:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787826257.386394:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.386569:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826257.387045:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346068736:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787826257.387065:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387081:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826257.387141:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005c1e933c reply state 000000004f2bbb9c transno 4294985299 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387155:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387162:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826257.387165:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826257.387168:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826257.387175:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387184:0:6527:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826257.387196:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18a:0x0] 00000004:00010000:0.0:1787826257.387203:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005c1e933c reply state 000000004f2bbb9c transno 4294985299 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387212:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387219:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:0.0:1787826257.387221:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826257.387223:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787826257.387230:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3c:0x0]: returning size 4096 00000004:00080000:0.0:1787826257.387242:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826257.387254:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f48d43 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48d4a expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387262:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f48d43 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48d4a expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387269:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f48d43 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48d4a expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387288:0:6527:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826257.387302:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826257.387307:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.387315:0:6527:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.387409:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222964:x1874663332489728:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 24634us (25517us total) trans 4294985299 rc 0/0 00000100:00100000:0.0:1787826257.387416:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68127 00000100:00100000:0.0:1787826257.392203:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826257.396065:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.397245:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346068992 00000100:00100000:1.0:1787826257.397252:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346068992 00000100:00100000:1.0:1787826257.397331:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7785 00000100:00100000:1.0:1787826257.397341:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6529:x1874663346068992:12345-0@lo:104: 00010000:00010000:1.0:1787826257.397357:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.397813:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826257.397824:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6529:x1874663346068992:12345-0@lo:104: Request processed in 483us (815us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.397833:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7785 00010000:00010000:0.0:1787826257.397900:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826257.397909:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.397918:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826257.397924:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826257.397934:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.397971:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.397977:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826257.398002:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.398009:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f48d51 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48d58 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.398066:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826257.398068:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826257.398070:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826257.398085:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346069120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826257.398145:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.398266:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b662a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6529:1874663346068992:0@lo:104: 00010000:00010000:2.0:1787826257.398302:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48d97 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.398873:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346069120 00000100:00100000:1.0:1787826257.398882:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346069120 00010000:00010000:1.0:1787826257.398917:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f48d58 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f48d51 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826257.398991:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21806 00000100:00100000:1.0:1787826257.399001:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346069120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826257.399017:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826257.401791:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f48d58 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f48d51 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.401818:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961d81900) 00010000:00010000:1.0:1787826257.401825:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48d97 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.401841:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48d97 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.401850:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48d97 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.401890:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826257.401997:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346069120:12345-0@lo:103:ldlm_bl.0 Request processed in 2995us (3849us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.402009:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21806 00000100:00100000:0.0:1787826257.405669:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346069120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826257.406193:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.406270:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332490368 00000100:00100000:2.0:1787826257.406281:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332490368 00000100:00100000:2.0:1787826257.406358:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68130 00000100:00100000:2.0:1787826257.406369:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b752d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222953:x1874663332490368:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826257.406402:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826257.406429:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f48da5 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800cbc expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.406509:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48dac lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.406525:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48dac lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.406538:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48dac lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826257.406782:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3c:0x0]: returning size 4096 00000004:00200000:2.0:1787826257.406789:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826257.406836:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f48da5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cbc expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.406856:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48dac lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cbc expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.406872:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:2.0:1787826257.406876:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826257.406880:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826257.406953:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b752d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222953:x1874663332490368:12345-192.168.203.56@tcp:101:bash.0 Request processed in 586us (765us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.406961:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68130 00000100:00100000:0.0:1787826257.409382:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.409525:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332490496 00000100:00100000:2.0:1787826257.409539:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332490496 00000100:00100000:2.0:1787826257.409681:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68131 00000100:00100000:2.0:1787826257.409694:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222953:x1874663332490496:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826257.409721:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826257.409749:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f48db3 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800cc3 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.409828:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48dba lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.409844:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48dba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.409857:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48dba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826257.409923:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18c:0x0]: returning size 4096 00000004:00200000:2.0:1787826257.409929:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826257.409976:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f48db3 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cc3 expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.410002:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48dba lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cc3 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.410011:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826257.410015:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826257.410019:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826257.410185:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222953:x1874663332490496:12345-192.168.203.56@tcp:101:bash.0 Request processed in 492us (816us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.410193:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68131 00000100:00100000:0.0:1787826257.413147:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332490624 00000100:00100000:0.0:1787826257.413160:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332490624 00000100:00100000:0.0:1787826257.413227:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68132 00000100:00100000:0.0:1787826257.413240:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:222953:x1874663332490624:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826257.413267:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826257.413302:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f48dc1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800cca expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.413379:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48dc8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.413396:0:24703:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f48d74 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48d6d expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.413406:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48dc8 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.413448:0:24703:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f48d74 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f48d6d expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826257.413467:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346069376:0@lo:104: 00000100:00100000:0.0:1787826257.413527:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826257.413588:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826257.413831:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346069376 00000100:00100000:2.0:1787826257.413838:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346069376 00000100:00100000:2.0:1787826257.413882:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7786 00000100:00100000:2.0:1787826257.413891:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346069376:12345-0@lo:104: 00010000:00010000:2.0:1787826257.413907:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.414005:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826257.414012:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346069376:12345-0@lo:104: Request processed in 123us (484us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826257.414020:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7786 00000100:00100000:0.0:1787826257.414223:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346069376:0@lo:104: 00010000:00010000:0.0:1787826257.414251:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48dc8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.414437:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826257.414448:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.414458:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826257.414464:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826257.414474:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.414515:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.414523:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826257.414632:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346069504:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826257.414673:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826257.415573:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 1/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.415582:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f48d6d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f48d74 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826257.415605:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826257.415607:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826257.415610:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826257.415959:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346069504:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826257.415994:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48dc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.416003:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48dc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826257.416099:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18a:0x0]: returning size 4096 00000004:00200000:0.0:1787826257.416266:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826257.416393:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f48dc1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cca expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.416413:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48dc8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cca expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.416422:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:0.0:1787826257.416425:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826257.416428:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826257.416536:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:222953:x1874663332490624:12345-192.168.203.56@tcp:101:bash.0 Request processed in 3300us (3580us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.416552:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68132 00000100:00100000:0.0:1787826257.419612:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.425533:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:0.0:1787826257.426609:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826257.434537:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.434846:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332491264 00000100:00100000:1.0:1787826257.434859:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332491264 00000100:00100000:1.0:1787826257.434932:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9282 00000100:00100000:1.0:1787826257.434944:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:222953:x1874663332491264:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:1.0:1787826257.435540:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:222953:x1874663332491264:12345-192.168.203.56@tcp:35:bash.0 Request processed in 595us (1015us total) trans 8589957283 rc 0/0 00000100:00100000:1.0:1787826257.435604:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9282 00000100:00100000:1.0:1787826257.442866:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826257.443232:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332491520 00000100:00100000:2.0:1787826257.443247:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332491520 00000100:00100000:2.0:1787826257.443352:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68134 00000100:00100000:2.0:1787826257.443365:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850359500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:222953:x1874663332491520:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826257.443404:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826257.443410:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826257.443446:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48dba lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cc3 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.443473:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826257.443479:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826257.443496:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f48df9 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800cdf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.443685:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48e00 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.443702:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48e00 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.443713:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48e00 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.443729:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.443736:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826257.443741:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826257.443922:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346069632:0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826257.444003:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826257.444059:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826257.444446:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346069632 00000100:00100000:1.0:1787826257.444456:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346069632 00000100:00100000:1.0:1787826257.444507:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23133 00000100:00100000:1.0:1787826257.444518:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:35094:x1874663346069632:12345-0@lo:1000:mdt00_011.0 00000100:00100000:1.0:1787826257.444757:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:35094:x1874663346069632:12345-0@lo:1000:mdt00_011.0 Request processed in 240us (752us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826257.444766:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23133 00000100:00100000:0.0:1787826257.455597:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332491648 00000100:00100000:0.0:1787826257.455612:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332491648 00000100:00100000:0.0:1787826257.455715:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30758 00000100:00100000:0.0:1787826257.455727:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:222953:x1874663332491648:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826257.455754:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826257.455784:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ce6 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826257.455822:0:17826:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4c6:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787826257.455853:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c6:0x0].0x0 since=8069s v=(0 0) 00010000:00010000:0.0:1787826257.455860:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826257.455875:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.455896:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826257.455914:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:0.0:1787826257.455993:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:222953:x1874663332491648:12345-192.168.203.56@tcp:101:bash.0 Request processed in 268us (566us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826257.456001:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30758 00000100:00100000:1.0:1787826257.458575:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.472481:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826257.472529:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:1.0:1787826258.093722:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826258.140532:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332492672 00000100:00100000:1.0:1787826258.140547:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332492672 00000100:00100000:1.0:1787826258.140621:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8735 00000100:00100000:1.0:1787826258.140634:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332492672:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826258.140673:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332492416 00000100:00100000:2.0:1787826258.140679:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332492416 00000100:00100000:2.0:1787826258.140718:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332492928 00000100:00100000:1.0:1787826258.140721:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332492672:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 89us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.140722:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332492928 00000100:00100000:2.0:1787826258.140743:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332492544 00000100:00100000:2.0:1787826258.140746:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332492544 00000100:00100000:2.0:1787826258.141361:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14191 00000100:00100000:2.0:1787826258.141375:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332492416:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826258.141424:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332492416:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 54us (1051us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.141432:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14191 00000100:00100000:2.0:1787826258.141456:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14192 00000100:00100000:2.0:1787826258.141463:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332492928:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826258.141494:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332492928:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 32us (1031us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.141499:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14192 00000100:00100000:2.0:1787826258.141510:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14193 00000100:00100000:2.0:1787826258.141515:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332492544:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826258.141544:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332492544:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (987us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.141550:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14193 00000100:00100000:2.0:1787826258.141574:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332492288 00000100:00100000:2.0:1787826258.141580:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332492288 00000100:00100000:2.0:1787826258.141613:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2200 00000100:00100000:2.0:1787826258.141621:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847253450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332492288:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826258.141648:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847253450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332492288:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (1121us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.141656:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2200 00000100:00100000:1.0:1787826258.143571:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8735 00000100:00100000:1.0:1787826258.147767:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826258.150343:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826258.150381:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826258.150459:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332493056 00000100:00100000:2.0:1787826258.150468:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332493056 00000100:00100000:2.0:1787826258.150524:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14194 00000100:00100000:2.0:1787826258.150535:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332493056:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826258.150592:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332493056:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 60us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.150599:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14194 00000100:00100000:2.0:1787826258.150640:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332493312 00000100:00100000:2.0:1787826258.150644:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332493312 00000100:00100000:2.0:1787826258.150672:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8738 00000100:00100000:2.0:1787826258.150679:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332493312:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826258.150710:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332493312:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 32us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.150717:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8738 00010000:00010000:2.0:1787826258.749825:0:164296:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:2.0:1787826258.749876:0:164296:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000100:00100000:2.0:1787826258.925710:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346069760:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.925816:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.925870:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346070272:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.925902:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.925924:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346070784:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.925954:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.925972:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346071296:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.925994:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.926034:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1787826258.926040:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346070144:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.926064:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.926083:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346070656:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.926144:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.926165:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346071168:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.926188:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.926222:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346069888:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.926252:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.926270:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346070400:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.929251:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.929312:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346070912:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.929355:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.929435:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346069888:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.929456:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1787826258.929465:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346070016:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.929496:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.929523:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346070528:0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.929711:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346070784 00000100:00100000:0.0:1787826258.929720:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346070784 00000100:00100000:0.0:1787826258.929783:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346071168 00000100:00100000:0.0:1787826258.929786:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346071168 00000100:00100000:0.0:1787826258.929807:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346070400 00000100:00100000:0.0:1787826258.929810:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346070400 00000100:00100000:0.0:1787826258.929832:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8739 00000100:00100000:0.0:1787826258.929841:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346070784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.929906:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346070784:12345-0@lo:400:kworker.0 Request processed in 66us (3950us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826258.929914:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8739 00000100:00100000:0.0:1787826258.929934:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8740 00000100:00100000:0.0:1787826258.929940:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346071168:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.929970:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346071168:12345-0@lo:400:kworker.0 Request processed in 31us (3782us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826258.929979:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8740 00000100:00100000:0.0:1787826258.929990:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8741 00000100:00100000:0.0:1787826258.929996:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346070400:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.930036:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346070400:12345-0@lo:400:kworker.0 Request processed in 40us (790us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826258.930042:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8741 00000100:00080000:0.0:1787826258.930061:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826258.930120:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346071168:0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.930138:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346070784:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932078:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.932142:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346071040:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932175:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826258.932219:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346070400:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932234:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346070912:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932294:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346069760 00000100:00100000:2.0:1787826258.932301:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346069760 00000100:00100000:2.0:1787826258.932346:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2201 00000100:00100000:2.0:1787826258.932354:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847253850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346069760:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932403:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847253850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346069760:12345-0@lo:400:kworker.0 Request processed in 50us (6588us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.932410:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2201 00000100:00100000:2.0:1787826258.932446:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346070272 00000100:00100000:2.0:1787826258.932451:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346070272 00000100:00100000:2.0:1787826258.932494:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346071296 00000100:00100000:2.0:1787826258.932498:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346071296 00000100:00100000:2.0:1787826258.932519:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346070144 00000100:00100000:2.0:1787826258.932523:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346070144 00000100:00100000:2.0:1787826258.932556:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346070656 00000100:00100000:2.0:1787826258.932559:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346070656 00000100:00100000:2.0:1787826258.932579:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346070016 00000100:00100000:2.0:1787826258.932582:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346070016 00000100:00100000:2.0:1787826258.932599:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346070528 00000100:00100000:2.0:1787826258.932602:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346070528 00000100:00100000:2.0:1787826258.932621:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14195 00000100:00100000:2.0:1787826258.932628:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346070272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932666:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346070272:12345-0@lo:400:kworker.0 Request processed in 38us (6762us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.932672:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14195 00000100:00100000:2.0:1787826258.932687:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14196 00000100:00100000:2.0:1787826258.932693:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346071296:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932721:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346071296:12345-0@lo:400:kworker.0 Request processed in 29us (6726us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.932730:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14196 00000100:00100000:2.0:1787826258.932741:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14197 00000100:00100000:2.0:1787826258.932746:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346070144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932772:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346070144:12345-0@lo:400:kworker.0 Request processed in 26us (6707us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.932777:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14197 00000100:00100000:2.0:1787826258.932788:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14198 00000100:00100000:2.0:1787826258.932794:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346070656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932821:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346070656:12345-0@lo:400:kworker.0 Request processed in 28us (6716us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.932826:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14198 00000100:00100000:2.0:1787826258.932837:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14199 00000100:00100000:2.0:1787826258.932843:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346070016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932875:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346070016:12345-0@lo:400:kworker.0 Request processed in 32us (3379us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.932881:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14199 00000100:00100000:2.0:1787826258.932893:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14200 00000100:00100000:2.0:1787826258.932899:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346070528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.932925:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346070528:12345-0@lo:400:kworker.0 Request processed in 27us (848us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.932931:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14200 00000100:00100000:2.0:1787826258.932952:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346071040 00000100:00100000:2.0:1787826258.932955:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346071040 00000100:00100000:2.0:1787826258.932982:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8742 00000100:00100000:2.0:1787826258.932988:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346071040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.933029:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346071040:12345-0@lo:400:kworker.0 Request processed in 41us (852us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826258.933035:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8742 00000100:00100000:2.0:1787826258.933152:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346070016:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.933165:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346070528:0@lo:400:kworker.0 00000100:00100000:2.0:1787826258.933175:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973828 gen 2 00000004:00080000:2.0:1787826258.933180:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f78435b2680, transno 4294973828 00000100:00100000:2.0:1787826258.933189:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346071040:0@lo:400:kworker.0 00000100:00080000:0.0:1787826258.933780:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787826258.933787:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787826258.933790:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826258.933842:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346070144:0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.933856:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346070656:0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.933867:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346069760:0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.933878:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346070272:0@lo:400:kworker.0 00000100:00100000:0.0:1787826258.933890:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346071296:0@lo:400:kworker.0 00000100:00100000:1.0:1787826259.176584:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826259.176659:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826259.176697:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332493568 00000100:00100000:1.0:1787826259.176711:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332493568 00000100:00100000:2.0:1787826259.179694:0:20179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332493440 00000100:00100000:2.0:1787826259.179717:0:20179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332493440 00002000:00100000:2.0:1787826259.179833:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x51b:0x0] (0->5) req@ffff9f7854624000 x1874663332493440/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:615/0 lens 496/0 e 0 to 0 dl 1787826270 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826259.179847:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7854624000 with x1874663332493440 ext(0->5) 00002000:00010000:2.0:1787826259.179855:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7854624000 00000100:00100000:2.0:1787826259.179898:0:20179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27624 00000100:00100000:2.0:1787826259.179911:0:20179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332493440:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:1.0:1787826259.181935:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30759 00000100:00100000:1.0:1787826259.181959:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332493568:12345-192.168.203.56@tcp:16:kworker.0 00000100:00080000:1.0:1787826259.182087:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826259.186412:0:20179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332493440:12345-192.168.203.56@tcp:4:bash.0 Request processed in 6486us (9828us total) trans 4294973831 rc 0/0 00000100:00100000:1.0:1787826259.186430:0:20179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27624 00002000:00100000:1.0:1787826259.186448:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x51b:0x0] (0->5) req@ffff9f7854624000 x1874663332493440/t4294973831(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:615/0 lens 496/448 e 0 to 0 dl 1787826270 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826259.186461:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7854624000 with x1874663332493440 ext(0->5) 00002000:00010000:1.0:1787826259.186469:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7854624000 00000001:00080000:2.0:1787826259.189458:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973831 is committed 00000001:00080000:2.0:1787826259.189487:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973830 is committed 00000100:00100000:2.0:1787826259.189652:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332493568:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 7693us (12982us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826259.189665:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30759 00000100:00100000:1.0:1787826259.216718:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826259.216787:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:0.0:1787826259.217197:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826259.220170:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332493824 00000100:00100000:2.0:1787826259.220201:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332493824 00000100:00100000:0.0:1787826259.226613:0:8449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332493696:12345-192.168.203.56@tcp:4:bash.0 Request processed in 9202us (9809us total) trans 4294976140 rc 0/0 00000100:00100000:0.0:1787826259.226641:0:8449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27625 00002000:00100000:0.0:1787826259.226659:0:8449:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x4c6:0x0] (0->5) req@ffff9f7851835f80 x1874663332493696/t4294976140(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:615/0 lens 496/448 e 0 to 0 dl 1787826270 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826259.226679:0:8449:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7851835f80 with x1874663332493696 ext(0->5) 00002000:00010000:0.0:1787826259.226688:0:8449:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7851835f80 00000100:00100000:2.0:1787826259.226729:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30760 00000100:00100000:2.0:1787826259.226749:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332493824:12345-192.168.203.56@tcp:16:kworker.0 00000001:00080000:2.0:1787826259.241518:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976140 is committed 00000001:00080000:2.0:1787826259.241545:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976139 is committed 00000100:00100000:2.0:1787826259.437236:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346071424:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826259.437461:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826259.438424:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346071424:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826259.948377:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826259.948412:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346071552:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826259.948507:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826259.948844:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826259.948856:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346071680:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826259.948891:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826259.948931:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346071552 00000100:00100000:1.0:1787826259.948940:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346071552 00000100:00100000:1.0:1787826259.948997:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346071680 00000100:00100000:1.0:1787826259.949001:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346071680 00000100:00100000:1.0:1787826259.949037:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23136 00000100:00100000:1.0:1787826259.949048:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346071552:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826259.949092:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443446784 avail 1297207296 00000100:00100000:1.0:1787826259.949181:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346071552:12345-0@lo:41:osp-pre-1-0.0 Request processed in 134us (673us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826259.949189:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23136 00000100:00100000:1.0:1787826259.949209:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23137 00000100:00100000:1.0:1787826259.949216:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346071680:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826259.949235:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442836480 avail 1296596992 00000100:00100000:1.0:1787826259.949284:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346071680:12345-0@lo:41:osp-pre-0-1.0 Request processed in 58us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826259.949291:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23137 00000100:00100000:1.0:1787826259.949467:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346071680:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826259.949509:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346071552:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826259.949816:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826259.949827:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346071936:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826259.949869:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826259.949906:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346071936 00000100:00100000:1.0:1787826259.949912:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346071936 00000100:00100000:1.0:1787826259.949950:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6184 00000100:00100000:1.0:1787826259.949957:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346071936:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826259.949974:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662430208 00000100:00100000:1.0:1787826259.950025:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346071936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (156us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826259.950032:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6184 00000100:00100000:1.0:1787826259.950056:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851886a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346071808:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826259.950083:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826259.950135:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346071808 00000100:00100000:1.0:1787826259.950139:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346071808 00000100:00100000:1.0:1787826259.950174:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6185 00000100:00100000:1.0:1787826259.950180:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346071808:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826259.950193:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683164160 00000100:00100000:1.0:1787826259.950226:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346071808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 46us (142us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826259.950233:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6185 00000100:00080000:1.0:1787826259.950248:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826259.950456:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346071936:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826259.950486:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976139 gen 2 00000004:00080000:1.0:1787826259.950492:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784d009c00, transno 4294976139 00000100:00100000:1.0:1787826259.950554:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851886a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346071808:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826259.954494:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826259.954526:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346072064:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826259.954621:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826259.955834:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346072064 00000100:00100000:1.0:1787826259.955851:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346072064 00000100:00100000:1.0:1787826259.955939:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6186 00000100:00100000:1.0:1787826259.955950:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346072064:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826259.955983:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662430208 00000100:00100000:1.0:1787826259.956078:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346072064:12345-0@lo:13:osp-pre-0-1.0 Request processed in 129us (1456us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826259.956086:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6186 00000100:00100000:1.0:1787826259.956162:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973830 gen 2 00000004:00080000:1.0:1787826259.956168:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7844295180, transno 4294973830 00000100:00100000:1.0:1787826259.956191:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346072064:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1787826259.956581:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787826260.158205:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826261.165450:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826261.779539:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826261.780144:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332493952 00000100:00100000:2.0:1787826261.780163:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332493952 00000100:00100000:2.0:1787826261.780251:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68135 00000100:00100000:2.0:1787826261.780276:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223134:x1874663332493952:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826261.780305:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826261.780332:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f48e23 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ced expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.780420:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f48e2a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.780438:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f48e2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.780452:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f48e2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826261.780515:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826261.780520:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826261.780546:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f48e2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.780575:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f48e23 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ced expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.780593:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f48e2a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ced expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.780603:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826261.780607:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826261.780611:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826261.780699:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223134:x1874663332493952:12345-192.168.203.56@tcp:101:cat.0 Request processed in 438us (1460us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826261.780706:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68135 00000100:00080000:1.0:1787826261.780945:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826261.802976:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826261.803073:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332494208 00000100:00100000:2.0:1787826261.803087:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332494208 00000100:00100000:2.0:1787826261.804761:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68137 00000100:00100000:2.0:1787826261.804779:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223134:x1874663332494208:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826261.804807:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826261.804846:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f48e4d lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800cfb expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.805366:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f48e54 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.805388:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f48e54 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.805406:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f48e54 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826261.805545:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3c:0x0]: returning size 4096 00000004:00200000:2.0:1787826261.805551:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826261.805625:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f48e4d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cfb expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.805648:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f48e54 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cfb expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.805657:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:2.0:1787826261.805660:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826261.805665:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826261.805765:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223134:x1874663332494208:12345-192.168.203.56@tcp:101:cat.0 Request processed in 991us (2799us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826261.805773:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68137 00000100:00080000:0.0:1787826261.808053:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826261.810328:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826261.811563:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332494336 00000100:00100000:0.0:1787826261.811582:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332494336 00000100:00100000:0.0:1787826261.811667:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68138 00000100:00100000:0.0:1787826261.811681:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223134:x1874663332494336:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826261.811711:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826261.811741:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f48e5b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d02 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826261.811820:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f48e62 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826261.811836:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f48e62 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826261.811849:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f48e62 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826261.811909:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18c:0x0]: returning size 4096 00000004:00200000:0.0:1787826261.811927:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826261.811973:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f48e5b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d02 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826261.811990:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f48e62 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d02 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826261.812013:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826261.812016:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826261.812021:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826261.812131:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223134:x1874663332494336:12345-192.168.203.56@tcp:101:cat.0 Request processed in 428us (1791us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826261.812137:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68138 00000100:00100000:0.0:1787826261.817435:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826261.817554:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332494464 00000100:00100000:1.0:1787826261.817569:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332494464 00000100:00100000:1.0:1787826261.817655:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68139 00000100:00100000:1.0:1787826261.817668:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223134:x1874663332494464:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826261.817697:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826261.817725:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48e69 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d09 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826261.817806:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826261.817824:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826261.817837:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826261.817896:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18a:0x0]: returning size 4096 00000004:00200000:1.0:1787826261.817902:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826261.817947:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48e69 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826261.817967:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826261.817977:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:1.0:1787826261.817980:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826261.817984:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:1.0:1787826261.818070:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223134:x1874663332494464:12345-192.168.203.56@tcp:101:cat.0 Request processed in 404us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826261.818078:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68139 00000100:00100000:1.0:1787826261.827250:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826261.827306:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332494720 00000100:00100000:2.0:1787826261.827321:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332494720 00000100:00100000:2.0:1787826261.827396:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68140 00000100:00100000:2.0:1787826261.827410:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223134:x1874663332494720:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826261.827439:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826261.827466:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f48e77 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d10 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.829102:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.829150:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.829171:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.829186:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.829193:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.829200:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:2.0:1787826261.829308:0:35080:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00010000:00010000:1.0:1787826262.188872:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826262.301259:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826262.301369:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332494848 00000100:00100000:2.0:1787826262.301387:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332494848 00000100:00100000:2.0:1787826262.301493:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68141 00000100:00100000:2.0:1787826262.301511:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223136:x1874663332494848:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826262.301542:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826262.301577:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48e8c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d17 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826262.301587:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787826262.301816:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.301835:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.301849:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826262.301914:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826262.301925:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826262.301955:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.301997:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48e8c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d17 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.302026:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d17 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.302037:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826262.302041:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826262.302045:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826262.302215:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223136:x1874663332494848:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 706us (962us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.302227:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68141 00000100:00100000:1.0:1787826262.308057:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826262.309493:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332494976 00000100:00100000:2.0:1787826262.309514:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332494976 00000100:00100000:2.0:1787826262.309617:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68142 00000100:00100000:2.0:1787826262.309631:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223136:x1874663332494976:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826262.309669:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826262.309696:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48e9a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d1e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309781:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309803:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309815:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309831:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309838:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309845:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309940:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309949:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.309956:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826262.309983:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310010:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310017:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310027:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310033:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310040:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826262.310045:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310052:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310066:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826262.310070:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826262.310075:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310081:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f48ea1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310096:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310102:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310157:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310163:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826262.310167:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310174:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310183:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8600) 00010000:00010000:2.0:1787826262.310187:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826262.310190:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310196:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48ea8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310246:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f48e9a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310262:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.310269:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:2.0:1787826262.310271:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826262.310274:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826262.310630:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223136:x1874663332494976:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 999us (2580us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.310640:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68142 00000100:00100000:1.0:1787826262.318521:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:0.0:1787826262.319791:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826262.325179:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826262.325285:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332495232 00000100:00100000:2.0:1787826262.325295:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332495232 00000100:00100000:2.0:1787826262.325345:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68144 00000100:00100000:2.0:1787826262.325355:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223136:x1874663332495232:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826262.325376:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826262.325399:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48ec4 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d2c expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325467:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325481:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325492:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325503:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325509:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325515:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325598:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48ed9 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325607:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48ed9 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325613:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48ed9 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826262.325652:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3d:0x0]: returning size 6 00000004:00200000:2.0:1787826262.325657:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826262.325676:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48ed9 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325687:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48ed9 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325695:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325701:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325706:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325713:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325719:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826262.325723:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325730:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325747:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826262.325750:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826262.325755:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325762:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f48ecb lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325781:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325787:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325793:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325798:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826262.325800:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325805:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325811:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x4ec94fd6 (ffff9f79462e8400) 00010000:00010000:2.0:1787826262.325813:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826262.325816:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325821:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f48ed2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325844:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48ec4 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d2c expref: 13 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325856:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48ed9 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d2c expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.325867:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:2.0:1787826262.325869:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826262.325871:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826262.325935:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223136:x1874663332495232:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 581us (765us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.325941:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68144 00000100:00100000:1.0:1787826262.341752:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826262.341841:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332495360 00000100:00100000:0.0:1787826262.341852:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332495360 00000100:00100000:0.0:1787826262.341913:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68145 00000100:00100000:0.0:1787826262.341924:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223136:x1874663332495360:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826262.341945:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826262.341967:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48ee0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d33 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342056:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342071:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342082:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342100:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342142:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342149:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342238:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48ef5 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342246:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48ef5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342252:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48ef5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826262.342304:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db2:0x0]: returning size 6 00000004:00200000:0.0:1787826262.342309:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826262.342327:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48ef5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342339:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48ef5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342347:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342353:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342363:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342369:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342374:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826262.342378:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342386:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342396:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:0.0:1787826262.342399:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826262.342404:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342410:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f48ee7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342423:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342429:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342435:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342440:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826262.342443:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342449:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342456:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x5a734ce9 (ffff9f79460a2a00) 00010000:00010000:0.0:1787826262.342458:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826262.342461:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342467:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f48eee lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.342493:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f48ee0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d33 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.343341:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48ef5 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d33 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.343356:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db2:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826262.343358:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826262.343361:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826262.343438:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223136:x1874663332495360:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1513us (1694us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.343446:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68145 00000100:00100000:0.0:1787826262.349329:0:6542:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332495744 00000100:00100000:0.0:1787826262.349340:0:6542:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332495744 00000100:00100000:0.0:1787826262.349411:0:6542:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1221 00000100:00100000:0.0:1787826262.349422:0:6542:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223136:x1874663332495744:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:0.0:1787826262.349476:0:6542:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826262.349503:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48ef5 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d33 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.349533:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db2:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826262.349545:0:6542:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826262.349627:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072192:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826262.349680:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826262.349737:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.349858:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332495488 00000100:00100000:2.0:1787826262.349863:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332495488 00000100:00100000:2.0:1787826262.349910:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10957 00000100:00100000:2.0:1787826262.349918:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:2394:x1874663332495488:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:2.0:1787826262.349928:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826262.349942:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48ed9 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d2c expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.349956:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:2.0:1787826262.349964:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826262.350023:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2394:x1874663332495488:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 106us (838us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.350031:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10957 00000100:00100000:2.0:1787826262.362809:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072192:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.362939:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072320:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.363299:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.363525:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346072320 00000100:00100000:1.0:1787826262.363533:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346072320 00000100:00100000:1.0:1787826262.363593:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23139 00000100:00100000:1.0:1787826262.363603:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346072320:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.363704:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346072320:12345-0@lo:1000:mdt_io00_002.0 Request processed in 101us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826262.363711:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23139 00000100:00100000:1.0:1787826262.364681:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346072448 00000100:00100000:1.0:1787826262.364690:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346072448 00000100:00100000:1.0:1787826262.364727:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23140 00000100:00100000:1.0:1787826262.364736:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346072448:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.364829:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346072448:12345-0@lo:1000:mdt_io00_002.0 Request processed in 93us (766us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826262.364836:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23140 00000100:00100000:0.0:1787826262.364912:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072448:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826262.364941:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e3c:0x0] 00000100:00100000:0.0:1787826262.364980:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ece00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072576:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826262.365027:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826262.365057:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.366576:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ece00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072576:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787826262.366642:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18c:0x0] 00000100:00100000:2.0:1787826262.366683:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072704:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.366732:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826262.366770:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848e219c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826262.366962:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346072704 00000100:00100000:1.0:1787826262.366970:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346072704 00000100:00100000:1.0:1787826262.367038:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23142 00000100:00100000:1.0:1787826262.367047:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79678fac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346072704:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.367164:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79678fac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346072704:12345-0@lo:1000:mdt_io00_002.0 Request processed in 118us (431us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826262.367171:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23142 00000100:00100000:2.0:1787826262.369237:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072704:0@lo:1000:mdt_io00_002.0 00000004:00020000:2.0:1787826262.369322:0:6542:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00000004:00020000:0.0:1787826262.369453:0:35080:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4460 00010000:00010000:0.0:1787826262.369532:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369550:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369562:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826262.369574:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db2:0x0]: returning size 6 00010000:00010000:0.0:1787826262.369681:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369690:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369697:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369704:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369709:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826262.369714:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369721:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369735:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:0.0:1787826262.369738:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826262.369742:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369749:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48e7e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369764:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369770:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369776:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369781:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826262.369785:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369790:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369797:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x5a734ce9 (ffff9f79460a2a00) 00010000:00010000:0.0:1787826262.369799:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826262.369802:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369807:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f48e85 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369853:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f48e77 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369867:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.369875:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db2:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826262.369877:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826262.369879:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826262.369956:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223134:x1874663332494720:12345-192.168.203.56@tcp:101:cat.0 Request processed in 542548us (542721us total) trans 8589957284 rc 0/0 00000100:00100000:0.0:1787826262.369966:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68140 00010000:00010000:2.0:1787826262.370812:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.370823:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.370834:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.370857:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072832:0@lo:101:mdt_io00_002.0 00000100:00100000:2.0:1787826262.370907:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826262.370944:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826262.371140:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346072832 00000100:00100000:0.0:1787826262.371148:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346072832 00000100:00100000:0.0:1787826262.371189:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23143 00000100:00100000:0.0:1787826262.371197:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346072832:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826262.371214:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826262.371228:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f48f11 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48f0a expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.371249:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 2/0,1 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.371271:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f48e62 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d02 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.371280:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f48f11 lrc: 5/0,0 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48f0a expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.371307:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f48e62 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d02 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.371320:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f48e62 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d02 expref: 8 pid: 6527 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826262.371330:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6533:1874663346072960:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826262.371595:0:6533:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:00080000:0.0:1787826262.371630:0:6533:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00000100:00100000:0.0:1787826262.371729:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826262.376682:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826262.376841:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6533:1874663346072960:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826262.376881:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f48f11 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48f0a expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.376899:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826262.376903:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826262.376908:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f48f11 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48f0a expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.376918:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826262.376983:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346072832:12345-0@lo:101:mdt_io00_002.0 Request processed in 5785us (6075us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.377010:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23143 00000100:00100000:1.0:1787826262.377805:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346072832:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826262.377829:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.377840:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.378023:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332495872 00000100:00100000:2.0:1787826262.378032:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332495872 00000100:00100000:2.0:1787826262.378136:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30761 00000100:00100000:2.0:1787826262.378149:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332495872:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826262.378167:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826262.378189:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f48f18 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d3a expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.378252:0:8445:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346073088:192.168.203.56@tcp:106: 00000100:00100000:2.0:1787826262.378338:0:8445:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548af00 going to sleep for 11 seconds 00000100:00080000:1.0:1787826262.379295:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1787826262.389982:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985301 is committed 00000020:00080000:2.0:1787826262.390011:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78513b3d80 refcount 2 committed 0 result 0 batchid 3381 00000020:00080000:2.0:1787826262.390016:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433da600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78513b3000 00000020:00080000:2.0:1787826262.390020:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1483 00000020:00080000:2.0:1787826262.390023:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433da400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7847b1fe00 00000020:00080000:2.0:1787826262.390025:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1165 00000020:00080000:2.0:1787826262.390042:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3379 batchid updated 00000001:00080000:2.0:1787826262.390049:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985298 is committed 00000020:00080000:2.0:1787826262.390055:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78434e4a80 refcount 2 committed 0 result 0 batchid 3380 00000020:00080000:2.0:1787826262.390058:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78434e4f00 00000020:00080000:2.0:1787826262.390061:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1482 00000020:00080000:2.0:1787826262.390064:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784b76b200 00000020:00080000:2.0:1787826262.390066:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1164 00000001:00080000:2.0:1787826262.390073:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985296 is committed 00010000:00010000:1.0:1787826262.390151:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390162:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390170:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390179:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826262.390185:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390192:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390209:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:1.0:1787826262.390212:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826262.390217:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390223:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f48d5f lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390237:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390244:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390249:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390255:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826262.390266:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390272:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390282:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:1.0:1787826262.390285:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826262.390292:0:3639:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f48f11 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48f0a expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390301:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390306:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48d66 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390321:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390327:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390333:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390337:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826262.390340:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390345:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390352:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x4ec94fd6 (ffff9f79462e8400) 00010000:00010000:1.0:1787826262.390354:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826262.390357:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.390361:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f48e07 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826262.390968:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6764 rpc_version 6764 00000020:00100000:0.0:1787826262.391012:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826262.391023:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78433ed500 x1874663346073344/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826262.391047:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346073344:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826262.391131:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826262.391161:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6000 going to sleep for 11 seconds 00002000:00010000:2.0:1787826262.391254:0:8445:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000401:0x51b:0x0] updating lvb size: 0 -> 6 00002000:00010000:2.0:1787826262.391266:0:8445:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000401:0x51b:0x0] updating lvb mtime: 0 -> 1787826256 00002000:00010000:2.0:1787826262.391268:0:8445:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000401:0x51b:0x0] updating lvb atime: 0 -> 1787826256 00002000:00010000:2.0:1787826262.391271:0:8445:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000401:0x51b:0x0] updating lvb ctime: 0 -> 1787826256 00002000:00010000:2.0:1787826262.391273:0:8445:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000401:0x51b:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:2.0:1787826262.391289:0:8445:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346073088:192.168.203.56@tcp:106: 00010000:00010000:2.0:1787826262.391322:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f48f18 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d3a expref: 6 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.391341:0:8445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f48f18 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d3a expref: 6 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.391356:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826262.391412:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332495872:12345-192.168.203.56@tcp:101:cat.0 Request processed in 13267us (14739us total) trans 0 rc 301/301 00000100:00100000:2.0:1787826262.391420:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30761 00000100:00100000:1.0:1787826262.392733:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346073344 00000100:00100000:1.0:1787826262.392746:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346073344 00000100:00100000:1.0:1787826262.392808:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23144 00000100:00100000:1.0:1787826262.392818:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346073344:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826262.393021:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346073344:12345-0@lo:1000:osp_up0-1.0 Request processed in 204us (1918us total) trans 8589957285 rc 0/0 00000100:00100000:1.0:1787826262.393029:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23144 00000100:00100000:2.0:1787826262.393926:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346073344:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826262.393943:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6765 rpc_version 6765 00000020:00100000:2.0:1787826262.393968:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826262.393976:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9ff100 x1874663346073600/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826262.394020:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346073600:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826262.394074:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826262.394097:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548af00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826262.395841:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826262.396996:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826262.397060:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332496128 00000100:00100000:2.0:1787826262.397065:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332496128 00000100:00100000:2.0:1787826262.397131:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30762 00000100:00100000:2.0:1787826262.397140:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:223134:x1874663332496128:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826262.397153:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826262.397170:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d41 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.397194:0:8759:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd41deae/0xe21309aa81f48df2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cd8 expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.397201:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x51b:0x0].0x0 since=5s v=(0 0) 00010000:00010000:2.0:1787826262.397205:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826262.397210:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 4/0,0 mode: --/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.397231:0:8759:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dd41deae/0xe21309aa81f48df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800cd8 expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.397244:0:8759:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dd41deae/0xe21309aa81f48df2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800cd8 expref: 5 pid: 35096 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.397254:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346073856:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826262.397303:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826262.399007:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346073728 00000100:00100000:1.0:1787826262.399020:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346073728 00000100:00100000:1.0:1787826262.399079:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7787 00000100:00100000:1.0:1787826262.399089:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6511:x1874663346073728:12345-0@lo:105: 00010000:00010000:1.0:1787826262.399129:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399141:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399150:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399158:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399170:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399179:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399203:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef000) 00000100:00100000:1.0:1787826262.399285:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6511:x1874663346073728:12345-0@lo:105: Request processed in 197us (2482us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826262.399293:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7787 00010000:00010000:1.0:1787826262.399322:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399331:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826262.399360:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18c:0x0] 00010000:00010000:1.0:1787826262.399399:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399406:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399415:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826262.399431:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346073984:0@lo:101:mdt_io00_002.0 00000100:00100000:1.0:1787826262.399465:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.399489:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826262.399504:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346073984 00000100:00100000:1.0:1787826262.399508:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346073984 00000100:00100000:1.0:1787826262.399545:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23146 00000100:00100000:1.0:1787826262.399552:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346073984:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826262.399566:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826262.399581:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48f2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48f26 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399603:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48f26 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399616:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48f26 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399624:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x4ec94fd6 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826262.399626:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826262.399630:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826262.399661:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346073984:12345-0@lo:101:mdt_io00_002.0 Request processed in 109us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826262.399667:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23146 00000100:00100000:1.0:1787826262.399700:0:6542:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985296 gen 2 00000004:00080000:1.0:1787826262.399706:0:6542:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985296 committed transno 4294985296 00000020:00080000:1.0:1787826262.399712:0:6542:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7845785600 refcount 2 committed 0 result 0 batchid 4294979073 00000020:00080000:1.0:1787826262.399716:0:6542:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433dad00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7845785a80 00000020:00080000:1.0:1787826262.399719:0:6542:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].929 00000020:00080000:1.0:1787826262.399721:0:6542:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433da700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7847b1fa00 00000020:00080000:1.0:1787826262.399724:0:6542:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].926 00000100:00100000:1.0:1787826262.399749:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346073984:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826262.399762:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48f2d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399770:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48f2d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826262.399783:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18c:0x0] 00010000:00010000:1.0:1787826262.399799:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399811:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399818:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399836:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.399847:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 10 pid: 6528 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826262.399858:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346074112:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826262.399914:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826262.411500:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826262.411812:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332496256 00000100:00100000:0.0:1787826262.411821:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332496256 00010000:00010000:0.0:1787826262.411879:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dd41deae/0xe21309aa81f48df2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800cd8 expref: 6 pid: 35096 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826262.411923:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21809 00000100:00100000:0.0:1787826262.411934:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332496256:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826262.411948:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826262.411972:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dd41deae/0xe21309aa81f48df2 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800cd8 expref: 6 pid: 35096 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.411985:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dd41deae/0xe21309aa81f48df2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800cd8 expref: 6 pid: 35096 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.412020:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826262.412025:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826262.412034:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.412045:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.412083:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826262.412096:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ee300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6513:1874663346074240:192.168.203.56@tcp:105: 00000100:00100000:0.0:1787826262.412203:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6000 going to sleep for 11 seconds 00010000:00010000:2.0:1787826262.412467:0:8759:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd41deae/0xe21309aa81f48df2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800cd8 expref: 6 pid: 35096 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.412483:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346073856:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826262.412506:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.412516:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826262.412547:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223134:x1874663332496128:12345-192.168.203.56@tcp:101:cat.0 Request processed in 15409us (15561us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.412555:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30762 00000100:00080000:1.0:1787826262.414851:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826262.428047:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332496384 00000100:00100000:2.0:1787826262.428057:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332496384 00010000:00010000:2.0:1787826262.428099:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 11 pid: 6528 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.428189:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21810 00000100:00100000:2.0:1787826262.428200:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332496384:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826262.428217:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826262.428224:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 11 pid: 6528 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.428237:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 11 pid: 6528 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.428274:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f48e70 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d09 expref: 11 pid: 6528 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.428292:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:2.0:1787826262.428297:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.428306:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.428313:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.428337:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826262.428368:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ee300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6513:1874663346074240:192.168.203.56@tcp:105: 00010000:00010000:0.0:1787826262.428402:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826262.428433:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332496256:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 16501us (16943us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.428441:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21809 00000100:00100000:2.0:1787826262.428566:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332496384:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 367us (723us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.428574:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21810 00010000:00010000:1.0:1787826262.429290:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.429305:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.429328:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.429339:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.429348:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826262.429423:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074368:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.429473:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.429509:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826262.430533:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074368:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.430597:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074496:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.430631:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.430870:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826262.433375:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23148 00000100:00100000:0.0:1787826262.433389:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346074496:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826262.433534:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346074496:12345-0@lo:1000:mdt_io00_002.0 Request processed in 148us (2901us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.433542:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23148 00000100:00100000:1.0:1787826262.434016:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074496:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.435179:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074624:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.435455:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.435542:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826262.437695:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074624:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.437798:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074752:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.437839:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.437956:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.437984:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346074752 00000100:00100000:2.0:1787826262.438006:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346074752 00000100:00100000:2.0:1787826262.438046:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23150 00000100:00100000:2.0:1787826262.438054:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346074752:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.438152:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346074752:12345-0@lo:1000:mdt_io00_002.0 Request processed in 98us (311us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.438158:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23150 00000100:00100000:1.0:1787826262.438236:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074752:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.438315:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074880:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.438342:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.438363:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.438736:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346074880 00000100:00100000:2.0:1787826262.438742:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346074880 00000100:00100000:2.0:1787826262.438768:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23151 00000100:00100000:2.0:1787826262.438776:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346074880:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.438858:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346074880:12345-0@lo:1000:mdt_io00_002.0 Request processed in 82us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.438865:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23151 00000100:00100000:1.0:1787826262.438937:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346074880:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826262.438964:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e3c:0x0] 00000100:00100000:0.0:1787826262.440605:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:1.0:1787826262.440915:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.440939:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.440949:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.441186:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.441204:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 9 pid: 35080 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826262.441222:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346075008:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826262.441566:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02da80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.441574:0:24158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332496512 00000100:00100000:2.0:1787826262.441582:0:24158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332496512 00002000:00100000:2.0:1787826262.441629:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_009: refresh rw locks for [0x280000401:0x51b:0x0] (0->4095) req@ffff9f78433ee300 x1874663332496512/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:618/0 lens 488/0 e 0 to 0 dl 1787826273 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826262.441638:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f78433ee300 with x1874663332496512 ext(0->4095) 00002000:00010000:2.0:1787826262.441644:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f78433ee300 00000100:00100000:2.0:1787826262.441679:0:24158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27626 00000100:00100000:2.0:1787826262.441689:0:24158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332496512:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:1.0:1787826262.445321:0:24158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332496512:12345-192.168.203.56@tcp:3:cat.0 Request processed in 3619us (4709us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787826262.445337:0:24158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27626 00002000:00100000:1.0:1787826262.445350:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_009: refresh rw locks for [0x280000401:0x51b:0x0] (0->4095) req@ffff9f78433ee300 x1874663332496512/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:618/0 lens 488/4536 e 0 to 0 dl 1787826273 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826262.445361:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f78433ee300 with x1874663332496512 ext(0->4095) 00002000:00010000:1.0:1787826262.445367:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f78433ee300 00000100:00100000:0.0:1787826262.456177:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826262.456311:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346075008:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826262.456349:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.456539:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332496640 00000100:00100000:2.0:1787826262.456548:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332496640 00010000:00010000:2.0:1787826262.456589:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 10 pid: 35080 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.456679:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21811 00000100:00100000:2.0:1787826262.456690:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2396:x1874663332496640:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826262.456713:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826262.456720:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 10 pid: 35080 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.456732:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 10 pid: 35080 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.456754:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f48efc lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d10 expref: 10 pid: 35080 timeout: 8174 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.456769:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db2:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:2.0:1787826262.456773:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.456782:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.456789:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.456814:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826262.456875:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332496640:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 186us (707us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.456882:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21811 00010000:00010000:1.0:1787826262.457206:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.457214:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.457269:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.457277:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.457285:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826262.457305:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346075136:0@lo:101:mdt_io00_002.0 00000100:00100000:1.0:1787826262.457350:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.457385:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02da80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826262.461594:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346075136 00000100:00100000:0.0:1787826262.461608:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346075136 00000100:00100000:0.0:1787826262.461687:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23152 00000100:00100000:0.0:1787826262.461698:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346075136:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826262.461849:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826262.461876:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f48f50 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48f49 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.461907:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f48f50 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48f49 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.461921:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f48f50 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48f49 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826262.461929:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f784b4a5700) 00010000:00010000:0.0:1787826262.461932:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787826262.461937:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826262.462042:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346075136:12345-0@lo:101:mdt_io00_002.0 Request processed in 346us (4691us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.462049:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23152 00000100:00100000:1.0:1787826262.462186:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346075136:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826262.462210:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48f50 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.462222:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48f50 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826262.462253:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e3d:0x0] 00000100:00100000:1.0:1787826262.462316:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346075264:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.462356:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.462384:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826262.462444:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346075264 00000100:00100000:0.0:1787826262.462449:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346075264 00000100:00100000:0.0:1787826262.462523:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23153 00000100:00100000:0.0:1787826262.462532:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346075264:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826262.462610:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346075264:12345-0@lo:1000:mdt_io00_002.0 Request processed in 79us (252us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.462617:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23153 00000100:00100000:2.0:1787826262.469656:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346075264:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787826262.469699:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18c:0x0] 00000100:00100000:2.0:1787826262.469768:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346075392:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.470807:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:0.0:1787826262.472681:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957287 is committed 00000020:00080000:0.0:1787826262.472692:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979073 batchid updated 00000020:00080000:1.0:1787826262.472795:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979073 00000004:00080000:1.0:1787826262.472864:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784d02d240 with version 1808 00000020:00080000:1.0:1787826262.472881:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979073 cancel update log [0x240000404:0x2:0x0].929: rc = 0 00000020:00080000:1.0:1787826262.472958:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979073 cancel update log [0x200000bd0:0x2:0x0].926: rc = 0 00000020:00080000:1.0:1787826262.472967:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979073 00000001:00080000:0.0:1787826262.473214:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957286 is committed 00000001:00080000:0.0:1787826262.473239:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957283 is committed 00010000:00010000:1.0:1787826262.473301:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f48ddd lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.473310:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f48ddd lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.473317:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f48ddd lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.473323:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826262.473328:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f48ddd lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.473335:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f48ddd lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.473349:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x5a734ce9 (ffff9f79460a2a00) 00010000:00010000:1.0:1787826262.473353:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826262.473357:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f48ddd lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826262.473363:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f48ddd lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.474815:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346075520 00000100:00100000:2.0:1787826262.474825:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346075520 00000100:00100000:2.0:1787826262.474889:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346075776 00000100:00100000:2.0:1787826262.474892:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346075776 00000100:00100000:2.0:1787826262.474924:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23155 00000100:00100000:2.0:1787826262.474934:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346075520:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.475057:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346075520:12345-0@lo:1000:mdt_io00_002.0 Request processed in 123us (2334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.475064:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23155 00000100:00100000:2.0:1787826262.475094:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23156 00000100:00100000:2.0:1787826262.475101:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346075776:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826262.475264:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346075776:12345-0@lo:1000:osp_up1-0.0 Request processed in 154us (932us total) trans 4294985302 rc 0/0 00000100:00100000:2.0:1787826262.475271:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23156 00000100:00100000:1.0:1787826262.475551:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346075776:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826262.477659:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346075904 00000100:00100000:1.0:1787826262.477667:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346075904 00000100:00100000:1.0:1787826262.477723:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23157 00000100:00100000:1.0:1787826262.477734:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346075904:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.477835:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346075904:12345-0@lo:1000:mdt_io00_002.0 Request processed in 103us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826262.477843:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23157 00000100:00100000:0.0:1787826262.478446:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346076032 00000100:00100000:0.0:1787826262.478452:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346076032 00000100:00100000:0.0:1787826262.478491:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23158 00000100:00100000:0.0:1787826262.478500:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346076032:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826262.478590:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346076032:12345-0@lo:1000:mdt_io00_002.0 Request processed in 91us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.478598:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23158 00000100:00100000:1.0:1787826262.478659:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076032:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826262.478681:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18c:0x0] 00000100:00100000:1.0:1787826262.478721:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076160:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.478746:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.478766:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.479171:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076160:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787826262.479194:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e3c:0x0] 00000100:00100000:2.0:1787826262.479239:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076288:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.479278:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826262.479293:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826262.479665:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346076288 00000100:00100000:0.0:1787826262.479671:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346076288 00000100:00100000:0.0:1787826262.479713:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23160 00000100:00100000:0.0:1787826262.479721:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346076288:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826262.479790:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346076288:12345-0@lo:1000:mdt_io00_002.0 Request processed in 70us (511us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.479797:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23160 00000100:00100000:1.0:1787826262.479971:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076288:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.480027:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076416:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.480054:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.480076:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02de40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.480431:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346076416 00000100:00100000:2.0:1787826262.480446:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346076416 00000100:00100000:2.0:1787826262.480486:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23161 00000100:00100000:2.0:1787826262.480495:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346076416:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826262.480573:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346076416:12345-0@lo:1000:mdt_io00_002.0 Request processed in 79us (517us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826262.480582:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23161 00000100:00100000:1.0:1787826262.480664:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076416:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.480756:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076544:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826262.480783:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.480804:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02de40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826262.480956:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346076544 00000100:00100000:0.0:1787826262.480961:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346076544 00000100:00100000:0.0:1787826262.480998:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23162 00000100:00100000:0.0:1787826262.481013:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346076544:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826262.481084:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6542:x1874663346076544:12345-0@lo:1000:mdt_io00_002.0 Request processed in 71us (300us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826262.481090:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23162 00000100:00100000:2.0:1787826262.481867:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346076544:0@lo:1000:mdt_io00_002.0 00000040:00080000:2.0:1787826262.482354:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].209, 64 off21568 00000004:00080000:2.0:1787826262.482364:0:6542:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8074694257975ns, next at 8075694255394ns, rc = 0 00000040:00080000:2.0:1787826262.482404:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].927, 904 off872288 00000040:00080000:2.0:1787826262.482465:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].930, 984 off970792 00000004:00080000:2.0:1787826262.482472:0:6542:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784548a300 with version 1809 00000004:00080000:1.0:1787826262.482529:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1809 rpc_version 1809 00000020:00100000:1.0:1787826262.482555:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826262.482563:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc31f80 x1874663346076800/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2584/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826262.482583:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346076800:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826262.482630:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826262.482662:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.482709:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346076800 00000100:00100000:2.0:1787826262.482716:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346076800 00000100:00100000:2.0:1787826262.482753:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23163 00000100:00100000:2.0:1787826262.482761:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346076800:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826262.483141:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+10:30909:x1874663346076800:12345-0@lo:1000:osp_up1-0.0 Request processed in 379us (508us total) trans 4294985303 rc 0/0 00000100:00100000:2.0:1787826262.483149:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23163 00000100:00100000:1.0:1787826262.483609:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346076800:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787826262.483750:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483762:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483771:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483776:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826262.483781:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483788:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483804:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db2:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:2.0:1787826262.483807:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826262.483811:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483817:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f48f42 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826262.483833:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e3d:0x0] 00010000:00010000:2.0:1787826262.483839:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48f50 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483848:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48f50 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483859:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483865:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483871:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483877:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826262.483879:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483884:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483897:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:2.0:1787826262.483899:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826262.483903:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483908:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f48f34 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826262.483923:0:6542:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000095cd1d2f reply state 0000000052cc4db6 transno 8589957288 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483935:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483941:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x5a734ce9 (ffff9f79460a2a00) 00010000:00010000:2.0:1787826262.483943:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826262.483946:0:6542:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:2.0:1787826262.483949:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18c:0x0] 00010000:00010000:2.0:1787826262.483953:0:6542:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483961:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.483966:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f48f0a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48f11 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484000:0:6542:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826262.484004:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18c:0x0] 00010000:00010000:2.0:1787826262.484010:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48f2d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484017:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48f2d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484025:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48f5e lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484031:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48f5e lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484037:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48f5e lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484042:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826262.484044:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48f5e lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484050:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48f5e lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484058:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7961d81500) 00010000:00010000:2.0:1787826262.484060:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826262.484063:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48f5e lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484077:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f48f5e lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484136:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48f65 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484152:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48f65 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484160:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48f65 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484168:0:6542:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48f65 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484173:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48f65 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826262.484176:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346076928:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826262.484179:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f48f65 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.484190:0:6542:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826262.484224:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826262.484391:0:6542:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223136:x1874663332495744:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 134969us (135293us total) trans 8589957288 rc 0/0 00000100:00100000:2.0:1787826262.484400:0:6542:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1221 00010000:00010000:1.0:1787826262.484668:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826262.491645:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346076928:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826263.225992:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826263.260502:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.260558:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.260590:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.260614:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826263.260615:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332497152 00000100:00100000:2.0:1787826263.260627:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332497152 00000100:00100000:1.0:1787826263.260632:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.260649:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.260667:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.260684:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826263.260684:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2202 00000100:00100000:2.0:1787826263.260694:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847251050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332497152:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826263.260705:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.260725:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332497408 00000100:00100000:1.0:1787826263.260729:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332497408 00000100:00100000:1.0:1787826263.260766:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332497792 00000100:00100000:2.0:1787826263.260766:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847251050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332497152:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 73us (207us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.260769:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332497792 00000100:00100000:2.0:1787826263.260774:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2202 00000100:00100000:1.0:1787826263.260787:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332497280 00000100:00100000:1.0:1787826263.260789:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332497280 00000100:00100000:1.0:1787826263.260807:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332497920 00000100:00100000:1.0:1787826263.260822:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332497920 00000100:00100000:2.0:1787826263.263639:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14201 00000100:00100000:2.0:1787826263.263658:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332497408:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826263.263719:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332497408:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 69us (3128us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.263727:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14201 00000100:00100000:2.0:1787826263.263753:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14202 00000100:00100000:2.0:1787826263.263762:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332497792:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826263.263794:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332497792:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 34us (3179us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.263801:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14202 00000100:00100000:2.0:1787826263.263816:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14203 00000100:00100000:2.0:1787826263.263824:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332497280:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826263.263853:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332497280:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (3202us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.263859:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14203 00000100:00100000:2.0:1787826263.263872:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14204 00000100:00100000:2.0:1787826263.263889:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332497920:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826263.263913:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332497920:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 35us (3228us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.263918:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14204 00000100:00080000:1.0:1787826263.270226:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826263.270237:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787826263.270240:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:1.0:1787826263.270253:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826263.658560:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332498304 00000100:00100000:1.0:1787826263.658570:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332498304 00000100:00100000:1.0:1787826263.658635:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68147 00000100:00100000:1.0:1787826263.658646:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223172:x1874663332498304:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826263.658664:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826263.658688:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48f6c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d4f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.658755:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f48f73 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.658769:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f48f73 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.658781:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f48f73 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826263.658869:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3c:0x0]: returning size 4096 00000004:00200000:1.0:1787826263.658874:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787826263.658893:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f48f73 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.658919:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48f6c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d4f expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.658943:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f48f73 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d4f expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.658953:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:1.0:1787826263.658956:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826263.658960:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826263.659046:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223172:x1874663332498304:12345-192.168.203.56@tcp:101:rm.0 Request processed in 402us (1151us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.659054:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68147 00000100:00100000:2.0:1787826263.663923:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826263.663964:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332498432 00000100:00100000:2.0:1787826263.663974:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332498432 00000100:00100000:2.0:1787826263.664039:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68148 00000100:00100000:2.0:1787826263.664049:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223172:x1874663332498432:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826263.664068:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826263.664091:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48f7a lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d56 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.664166:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.664180:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.664191:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826263.664233:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18c:0x0]: returning size 4096 00000004:00200000:2.0:1787826263.664238:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826263.666318:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48f7a lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.666343:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.666352:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f7967ac8f00) 00010000:00010000:2.0:1787826263.666355:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826263.666358:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826263.666421:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223172:x1874663332498432:12345-192.168.203.56@tcp:101:rm.0 Request processed in 2373us (2508us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.666429:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68148 00000100:00100000:1.0:1787826263.672865:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332498560 00000100:00100000:1.0:1787826263.672880:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332498560 00000100:00100000:1.0:1787826263.673498:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68149 00000100:00100000:1.0:1787826263.673511:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223172:x1874663332498560:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826263.673532:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826263.673558:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48f88 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d5d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.673669:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48f8f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.673683:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48f8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.673695:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48f8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826263.673739:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18a:0x0]: returning size 4096 00000004:00200000:1.0:1787826263.673757:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826263.673796:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f48f88 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d5d expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.673812:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48f8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d5d expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.673821:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826263.673825:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826263.673828:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826263.673897:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223172:x1874663332498560:12345-192.168.203.56@tcp:101:rm.0 Request processed in 389us (5197us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.673916:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68149 00000100:00100000:2.0:1787826263.683165:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332498816 00000100:00100000:2.0:1787826263.683174:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332498816 00000100:00100000:2.0:1787826263.683273:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68150 00000100:00100000:2.0:1787826263.683284:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223172:x1874663332498816:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826263.683305:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826263.683327:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48f96 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d64 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826263.683499:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3c:0x0]: returning size 4096 00010000:00010000:2.0:1787826263.683607:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48f96 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d64 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.683621:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac8b00) 00010000:00010000:2.0:1787826263.683624:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826263.683629:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f48f96 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d64 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.683647:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826263.683708:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223172:x1874663332498816:12345-192.168.203.56@tcp:101:rm.0 Request processed in 426us (798us total) trans 4294985304 rc 301/301 00000100:00100000:2.0:1787826263.683715:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68150 00000100:00100000:2.0:1787826263.713703:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.713842:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332498944 00000100:00100000:1.0:1787826263.713853:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332498944 00000100:00100000:1.0:1787826263.713917:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9285 00000100:00100000:1.0:1787826263.713929:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223172:x1874663332498944:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826263.725492:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223172:x1874663332498944:12345-192.168.203.56@tcp:37:rm.0 Request processed in 11555us (11786us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.725632:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9285 00000100:00100000:0.0:1787826263.734566:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.736651:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332499072 00000100:00100000:1.0:1787826263.736665:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332499072 00000100:00100000:1.0:1787826263.736732:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9286 00000100:00100000:1.0:1787826263.736744:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223172:x1874663332499072:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826263.740769:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223172:x1874663332499072:12345-192.168.203.56@tcp:37:rm.0 Request processed in 4016us (6270us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.740782:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9286 00000100:00100000:0.0:1787826263.753182:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.753407:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332499328 00000100:00100000:1.0:1787826263.753419:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332499328 00000100:00100000:1.0:1787826263.753481:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9287 00000100:00100000:1.0:1787826263.753492:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223172:x1874663332499328:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:1.0:1787826263.753930:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1166, 344 off1059304 00000040:00080000:1.0:1787826263.753996:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1484, 424 off1497800 00000004:00080000:1.0:1787826263.754002:0:6530:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d02d9c0 with version 6766 00000004:00080000:2.0:1787826263.757412:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6766 rpc_version 6766 00000020:00100000:2.0:1787826263.757463:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826263.757475:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7848cfed80 x1874663346077184/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826263.757511:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346077184:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826263.757591:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826263.757628:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.759001:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957286 gen 2 00000004:00080000:2.0:1787826263.759011:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957285 committed transno 8589957286 00000004:00080000:2.0:1787826263.759033:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957286 committed transno 8589957286 00000100:00100000:2.0:1787826263.759055:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f7848cfed80 x1874663346077184/t8589957289(8589957289) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826279 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826263.759096:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346077184:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826263.759409:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223172:x1874663332499328:12345-192.168.203.56@tcp:35:rm.0 Request processed in 5917us (6234us total) trans 4294985305 rc 0/0 00000100:00100000:0.0:1787826263.759419:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9287 00000100:00100000:0.0:1787826263.778295:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826263.778418:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332499584 00000100:00100000:2.0:1787826263.778428:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332499584 00000100:00100000:2.0:1787826263.778485:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68151 00000100:00100000:2.0:1787826263.778494:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223172:x1874663332499584:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826263.778530:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826263.778554:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f48fa4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d6b expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.778719:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48fab lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.778733:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48fab lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.778745:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48fab lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826263.778758:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3c:0x0]: returning size 4096 00010000:00010000:2.0:1787826263.778855:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f48fa4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d6b expref: 13 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.778877:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48fab lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d6b expref: 12 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.778891:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:2.0:1787826263.778894:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1787826263.778897:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826263.778958:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223172:x1874663332499584:12345-192.168.203.56@tcp:101:rm.0 Request processed in 464us (675us total) trans 4294985306 rc 0/0 00000100:00100000:2.0:1787826263.778965:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68151 00000100:00100000:0.0:1787826263.787138:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.787365:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332499712 00000100:00100000:1.0:1787826263.787461:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332499712 00000100:00100000:1.0:1787826263.787641:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68152 00000100:00100000:1.0:1787826263.787653:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223172:x1874663332499712:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:1.0:1787826263.787705:0:35092:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345594368 00000100:00100000:1.0:1787826263.787781:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223172:x1874663332499712:12345-192.168.203.56@tcp:41:rm.0 Request processed in 129us (675us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.787788:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68152 00000100:00100000:0.0:1787826263.790903:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826263.793412:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332499840 00000100:00100000:2.0:1787826263.793427:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332499840 00000100:00100000:2.0:1787826263.793509:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68153 00000100:00100000:2.0:1787826263.793521:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223172:x1874663332499840:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826263.793543:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826263.793564:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48fb9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d72 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793637:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793651:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793665:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793677:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793685:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793692:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793754:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f48fce lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793762:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f48fce lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793770:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f48fce lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826263.793789:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f48fce lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793803:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793809:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793815:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793822:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793828:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826263.793832:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793838:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793852:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:2.0:1787826263.793855:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826263.793859:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793866:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f48fc0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793879:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793885:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793891:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793896:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826263.793899:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793905:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793912:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x5a734ce9 (ffff9f79460a2a00) 00010000:00010000:2.0:1787826263.793915:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826263.793918:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793923:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f48fc7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793949:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f48fb9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d72 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793963:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f48fce lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d72 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.793971:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f7967ac8000) 00010000:00010000:2.0:1787826263.793973:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826263.793975:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826263.794050:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223172:x1874663332499840:12345-192.168.203.56@tcp:101:rm.0 Request processed in 531us (3159us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.794058:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68153 00000100:00100000:1.0:1787826263.801030:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826263.801165:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332499968 00000100:00100000:2.0:1787826263.801174:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332499968 00000100:00100000:0.0:1787826263.805186:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68154 00000100:00100000:0.0:1787826263.805208:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967912680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223172:x1874663332499968:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826263.805237:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826263.805270:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48fd5 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d79 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.805331:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48fdc lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.805345:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48fdc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.805357:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48fdc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826263.805407:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3d:0x0]: returning size 6 00000004:00200000:0.0:1787826263.805412:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:0.0:1787826263.805446:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f48fd5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d79 expref: 14 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.805461:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f48fdc lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d79 expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.805469:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f784b4a5700) 00010000:00010000:0.0:1787826263.805472:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826263.805476:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826263.805561:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967912680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223172:x1874663332499968:12345-192.168.203.56@tcp:101:rm.0 Request processed in 362us (4540us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826263.805568:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68154 00000100:00100000:0.0:1787826263.822555:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826263.822610:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826263.822644:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332500096 00000100:00100000:0.0:1787826263.822655:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332500096 00000100:00100000:0.0:1787826263.822723:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10959 00000100:00100000:0.0:1787826263.822735:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332500096:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826263.822751:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826263.822779:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f48fce lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d72 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.822800:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f7967ac8000) 00010000:00010000:0.0:1787826263.822813:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826263.822869:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332500096:12345-192.168.203.56@tcp:103:rm.0 Request processed in 136us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826263.822878:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10959 00000100:00100000:0.0:1787826263.839832:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346077440:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826263.839874:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f48fea lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48fe3 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.839889:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:0.0:1787826263.839891:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826263.839896:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f48fea lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48fe3 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.839905:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:0.0:1787826263.839991:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346077312:12345-0@lo:101:mdt00_008.0 Request processed in 13825us (14936us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826263.840000:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23165 00000100:00100000:1.0:1787826263.840502:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704700 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346077312:0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826263.840524:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.840537:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.848963:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.849022:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332500480 00000100:00100000:1.0:1787826263.849033:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332500480 00010000:00010000:1.0:1787826263.849076:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48f8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d5d expref: 10 pid: 6528 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.849138:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21812 00000100:00100000:1.0:1787826263.849149:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332500480:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826263.849163:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826263.849170:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48f8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d5d expref: 10 pid: 6528 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.849185:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48f8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d5d expref: 10 pid: 6528 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.849209:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f48f8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d5d expref: 10 pid: 6528 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.849225:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826263.849231:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f48fea lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48fe3 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.849243:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f48fea lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48fe3 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.849927:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f48fea lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48fe3 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.849971:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f48fea lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48fe3 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.849996:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346077568:0@lo:105: 00000100:00100000:1.0:1787826263.850056:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.850128:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02da80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.853387:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346077568 00000100:00100000:2.0:1787826263.853402:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346077568 00000100:00100000:2.0:1787826263.853459:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7788 00000100:00100000:2.0:1787826263.853471:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346077568:12345-0@lo:105: 00010000:00010000:2.0:1787826263.853490:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.853503:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.853513:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.853528:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.853536:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.853545:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.853572:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68a00) 00000100:00100000:2.0:1787826263.853651:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346077568:12345-0@lo:105: Request processed in 182us (3596us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.853659:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7788 00000100:00100000:2.0:1787826263.853707:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346077568:0@lo:105: 00010000:00010000:2.0:1787826263.853728:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826263.853777:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332500480:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 4628us (4822us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.853786:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21812 00010000:00010000:1.0:1787826263.857780:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.857800:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826263.857828:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18a:0x0] 00010000:00010000:1.0:1787826263.857944:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.857952:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.857964:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.858001:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346077696:0@lo:101:mdt00_008.0 00000100:00100000:1.0:1787826263.858066:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.858100:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.858141:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346077696 00000100:00100000:1.0:1787826263.858148:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346077696 00000100:00100000:1.0:1787826263.858195:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23166 00000100:00100000:1.0:1787826263.858204:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346077696:12345-0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826263.858217:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826263.858232:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48ff1 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.859002:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 2/0,1 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.859017:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 4/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48ff1 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.859035:0:6533:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00080000:00080000:1.0:1787826263.859066:0:6533:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00010000:00010000:1.0:1787826263.859147:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48ff1 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.859158:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x5a734ce9 (ffff9f79460a2a00) 00010000:00010000:1.0:1787826263.859162:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826263.859166:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48ff1 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.859175:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826263.859235:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35080:x1874663346077696:12345-0@lo:101:mdt00_008.0 Request processed in 1032us (1170us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.859243:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23166 00000100:00100000:1.0:1787826263.859430:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346077696:0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826263.859447:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.859457:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863037:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48f2d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826263.863059:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48f2d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863180:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48f2d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863188:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826263.863210:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecec00/0xe21309aa81f48f26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48f2d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863230:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48f50 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826263.863238:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48f50 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863254:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48f50 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863260:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826263.863276:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf600/0xe21309aa81f48f49 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48f50 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826263.863289:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957289 is committed 00000001:00080000:1.0:1787826263.863302:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957288 is committed 00000004:00080000:1.0:1787826263.863307:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 208 committed 00000100:00100000:1.0:1787826263.863345:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346077824:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826263.863428:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.863483:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346077952:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826263.863523:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826263.863558:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863567:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863573:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863580:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826263.863584:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863591:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863604:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x5a734ce9 (ffff9f79460a2a00) 00010000:00010000:1.0:1787826263.863607:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826263.863612:0:3639:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48ff1 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863623:0:3639:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48ff1 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863631:0:3639:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f48ff1 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.863660:0:3639:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48ff1 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.863672:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3639:1874663346078080:0@lo:105: 00000100:00100000:1.0:1787826263.864564:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.878581:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346077824 00000100:00100000:1.0:1787826263.878591:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346077824 00000100:00100000:1.0:1787826263.878641:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346077952 00000100:00100000:1.0:1787826263.878643:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346077952 00000100:00100000:1.0:1787826263.878664:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10960 00000100:00100000:1.0:1787826263.878674:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7947dffc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3655:x1874663346077824:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826263.878688:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826263.878723:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f48f2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48f26 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.878745:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x4ec94fd6 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826263.878757:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826263.878830:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dffc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346077824:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 157us (15402us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.878837:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10960 00000100:00100000:1.0:1787826263.878898:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10961 00000100:00100000:1.0:1787826263.878905:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346077952:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826263.878911:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826263.878922:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f48f50 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48f49 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.878933:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f784b4a5700) 00010000:00010000:1.0:1787826263.878941:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826263.878975:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346077952:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 57us (15438us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.878992:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10961 00000100:00100000:1.0:1787826263.879016:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346078080 00000100:00100000:1.0:1787826263.879019:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346078080 00000100:00100000:1.0:1787826263.879047:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7789 00000100:00100000:1.0:1787826263.879052:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:3639:x1874663346078080:12345-0@lo:105: 00010000:00010000:1.0:1787826263.879062:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.879074:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.879083:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.879091:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 5/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.879099:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.879127:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.879151:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf800) 00000100:00100000:1.0:1787826263.879178:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:3639:x1874663346078080:12345-0@lo:105: Request processed in 125us (14617us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.879187:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7789 00000100:00100000:1.0:1787826263.879227:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346077952:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826263.879248:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826263.879256:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346078208:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787826263.879288:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.879332:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346077824:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826263.879353:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.879362:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826263.879381:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18a:0x0] 00000100:00100000:1.0:1787826263.879415:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346078336:0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826263.879443:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.879458:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02da80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.879472:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346078336 00000100:00100000:1.0:1787826263.879475:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346078336 00000100:00100000:1.0:1787826263.879502:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23167 00000100:00100000:1.0:1787826263.879509:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346078336:12345-0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826263.879568:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346078336:12345-0@lo:1000:mdt00_008.0 Request processed in 60us (124us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.879574:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23167 00000100:00100000:1.0:1787826263.879593:0:35080:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957289 gen 2 00000004:00080000:1.0:1787826263.879598:0:35080:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957289 committed transno 8589957289 00000100:00100000:1.0:1787826263.879616:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346078336:0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826263.879653:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346078464:0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826263.879671:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.879682:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02da80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.879695:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346078464 00000100:00100000:1.0:1787826263.879699:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346078464 00000100:00100000:1.0:1787826263.879722:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23168 00000100:00100000:1.0:1787826263.879730:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346078464:12345-0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826263.879794:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346078464:12345-0@lo:1000:mdt00_008.0 Request processed in 64us (121us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.879800:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23168 00000100:00100000:1.0:1787826263.879828:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346078464:0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826263.879863:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346078592:0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826263.879886:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.879899:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02da80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.879909:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346078592 00000100:00100000:1.0:1787826263.879912:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346078592 00000100:00100000:1.0:1787826263.879930:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23169 00000100:00100000:1.0:1787826263.879936:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346078592:12345-0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826263.880017:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346078592:12345-0@lo:1000:mdt00_008.0 Request processed in 80us (129us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826263.880023:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23169 00000100:00100000:1.0:1787826263.880045:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346078592:0@lo:1000:mdt00_008.0 00010000:00010000:1.0:1787826263.880091:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880123:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880134:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.880146:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346078720:0@lo:101:mdt00_008.0 00000100:00100000:1.0:1787826263.880170:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.880185:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.880200:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346078720 00000100:00100000:1.0:1787826263.880203:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346078720 00000100:00100000:1.0:1787826263.880228:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23170 00000100:00100000:1.0:1787826263.880236:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:35080:x1874663346078720:12345-0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826263.880246:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826263.880262:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f49006 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f48fff expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880283:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f49006 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f48fff expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880296:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f49006 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f48fff expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880304:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826263.880307:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826263.880310:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826263.880339:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35080:x1874663346078720:12345-0@lo:101:mdt00_008.0 Request processed in 104us (167us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.880348:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23170 00000100:00100000:1.0:1787826263.880379:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346078720:0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826263.880394:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880404:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880426:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880436:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880443:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787826263.880703:0:35080:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].218, 64 off22144 00000004:00080000:1.0:1787826263.880715:0:35080:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8076092608105ns, next at 8077092605253ns, rc = 0 00000040:00080000:1.0:1787826263.880747:0:35080:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1167, 384 off1059688 00000040:00080000:1.0:1787826263.880797:0:35080:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1485, 464 off1498264 00000004:00080000:1.0:1787826263.880803:0:35080:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d02d780 with version 6767 00000100:00100000:1.0:1787826263.880894:0:3639:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.880911:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3639:1874663346078080:0@lo:105: 00010000:00010000:1.0:1787826263.880927:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.880934:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f48f3b lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826263.881156:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6767 rpc_version 6767 00000020:00100000:2.0:1787826263.881181:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826263.881189:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7848cfca80 x1874663346078976/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826263.881205:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfca80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346078976:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826263.881241:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826263.881264:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.882825:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346078976 00000100:00100000:1.0:1787826263.882837:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346078976 00000100:00100000:1.0:1787826263.886079:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23171 00000100:00100000:1.0:1787826263.886102:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346078976:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826263.886764:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346078976:12345-0@lo:1000:osp_up0-1.0 Request processed in 605us (5453us total) trans 8589957290 rc 0/0 00000100:00100000:1.0:1787826263.886775:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23171 00000100:00100000:1.0:1787826263.887357:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346078208 00000100:00100000:1.0:1787826263.887366:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346078208 00000100:00100000:1.0:1787826263.887417:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30764 00000100:00100000:1.0:1787826263.887427:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346078208:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1787826263.887452:0:17826:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1307 count 1 00010000:00010000:1.0:1787826263.887478:0:17826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826263.887895:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787826263.889491:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.889529:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51b:0x0].0x0 since=2s v=(1 0) 00010000:00010000:1.0:1787826263.889535:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51b:0x0].0x0 since=0s v=(0 1) 00010000:00010000:1.0:1787826263.889540:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.889592:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.889616:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 5 pid: 8759 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.889634:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346079104:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826263.889733:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.890142:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfca80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346078976:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826263.891386:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891408:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891421:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891430:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891447:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891453:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891474:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826263.891507:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891521:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891533:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891538:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826263.891543:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891551:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891563:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f7967ac1300) 00010000:00010000:1.0:1787826263.891566:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826263.891571:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891577:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4900d lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891591:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891599:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891605:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891620:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787826263.891636:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18a:0x0] 00010000:00010000:1.0:1787826263.891644:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891650:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891657:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891672:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787826263.891675:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18a:0x0] 00010000:00010000:1.0:1787826263.891680:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.891688:0:35080:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.892171:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223172:x1874663332500352:12345-192.168.203.56@tcp:36:rm.0 Request processed in 67389us (69624us total) trans 4294985307 rc 0/0 00000100:00100000:1.0:1787826263.892181:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68155 00010000:00010000:1.0:1787826263.892220:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787826263.892226:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892232:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826263.892236:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892243:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892253:0:30445:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f7967ac1300) 00010000:00010000:1.0:1787826263.892256:0:30445:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826263.892259:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892264:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4901b lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e3d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892280:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826263.892283:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892289:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826263.892293:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826263.892300:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892346:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892353:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826263.892373:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892380:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f48fff lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49006 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892398:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826263.892401:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892406:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826263.892410:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826263.892417:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892434:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892451:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826263.892458:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892465:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f48fe3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48fea expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.892478:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826263.892480:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826263.892482:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826263.892487:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826263.892491:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826263.893635:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346079232:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826263.893813:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.893855:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826263.894100:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967911f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346079360:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826263.894143:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826263.894187:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346079232 00000100:00100000:1.0:1787826263.894194:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346079232 00000100:00100000:1.0:1787826263.894240:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346079360 00000100:00100000:1.0:1787826263.894244:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346079360 00000100:00100000:0.0:1787826263.895173:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10962 00000100:00100000:0.0:1787826263.895188:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346079232:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826263.895205:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826263.895233:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f49006 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48fff expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.895250:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3d:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:0.0:1787826263.895261:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826263.895315:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346079232:12345-0@lo:103:ldlm_bl.0 Request processed in 131us (1503us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826263.895324:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10962 00000100:00100000:0.0:1787826263.895346:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10963 00000100:00100000:0.0:1787826263.895353:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346079360:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826263.895360:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826263.895374:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f48fea lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48fe3 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.895385:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:0.0:1787826263.895391:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826263.895415:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346079360:12345-0@lo:103:ldlm_bl.0 Request processed in 63us (1272us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826263.895421:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10963 00000100:00100000:1.0:1787826263.895991:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967911f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346079360:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826263.896005:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346079232:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826263.899994:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346079104:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826263.900027:0:17826:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826263.902487:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332500992 00000100:00100000:2.0:1787826263.902503:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332500992 00000100:00100000:2.0:1787826263.902587:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9288 00000100:00100000:2.0:1787826263.902600:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223172:x1874663332500992:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:2.0:1787826263.902852:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:223172:x1874663332500992:12345-192.168.203.56@tcp:35:rm.0 Request processed in 249us (2949us total) trans 4294985308 rc 0/0 00000100:00100000:2.0:1787826263.902861:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9288 00000100:00100000:0.0:1787826263.905946:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332500736 00000100:00100000:0.0:1787826263.905961:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332500736 00000100:00100000:0.0:1787826263.906047:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332500608 00000100:00100000:0.0:1787826263.906050:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332500608 00010000:00010000:0.0:1787826263.906072:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826263.906131:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21813 00000100:00100000:0.0:1787826263.906142:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332500608:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826263.906158:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826263.906167:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.906167:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10964 00000100:00100000:1.0:1787826263.906184:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332500736:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826263.906187:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.906198:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826263.906218:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f590973/0xe21309aa81f48f1f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d41 expref: 6 pid: 8759 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.906228:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f48e46 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cf4 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.906238:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51b:0x0].0x0 since=0s v=(0 1) 00010000:00010000:0.0:1787826263.906243:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51b:0x0].0x0 since=0s v=(0 1) 00010000:00010000:1.0:1787826263.906247:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:0.0:1787826263.906251:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.906253:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826263.906265:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.906315:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332500736:12345-192.168.203.56@tcp:103:rm.0 Request processed in 136us (6501us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.906334:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10964 00000100:00100000:1.0:1787826263.906783:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826263.906825:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332501248 00000100:00100000:2.0:1787826263.906832:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332501248 00000100:00100000:2.0:1787826263.906876:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68156 00000100:00100000:2.0:1787826263.906886:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d814a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:223172:x1874663332501248:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:2.0:1787826263.906943:0:11629:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826263.906972:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f48fab lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d6b expref: 14 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.907005:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f48f73 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d4f expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.907018:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f48e54 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800cfb expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.907027:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:2.0:1787826263.907030:0:11629:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826263.907074:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.907088:0:11629:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.907098:0:11629:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826263.907174:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346079488:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787826263.907223:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826263.907258:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bcd80 going to sleep for 11 seconds 00010000:00010000:0.0:1787826263.909332:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826263.909399:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332500608:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3254us (9538us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826263.909402:0:17826:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826263.909415:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21813 00010000:00010000:1.0:1787826263.909421:0:17826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.909434:0:17826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.909543:0:17826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.909568:0:17826:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826263.909624:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826263.909632:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.909642:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826263.909646:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.909656:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.909676:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.909684:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002fc2162/0xe21309aa81f49014 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.909719:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826263.909721:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826263.909723:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826263.909949:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346078208:12345-0@lo:6:osp-syn-0-0.0 Request processed in 22522us (30659us total) trans 4294973832 rc 0/0 00000100:00100000:1.0:1787826263.909958:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30764 00000004:00080000:0.0:1787826263.910399:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f79465cb100/2, rc 0, transno 4294973832 00000100:00100000:0.0:1787826263.910407:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346078208:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787826263.912284:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826263.912341:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826263.912432:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332501376 00000100:00100000:0.0:1787826263.912440:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332501376 00010000:00010000:0.0:1787826263.912478:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d17 expref: 8 pid: 6529 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826263.912509:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332501504 00000100:00100000:0.0:1787826263.912512:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332501504 00010000:00010000:0.0:1787826263.912523:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f48e2a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ced expref: 8 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826263.912539:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21814 00000100:00100000:0.0:1787826263.912547:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332501376:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826263.912559:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826263.912565:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d17 expref: 8 pid: 6529 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.912580:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d17 expref: 8 pid: 6529 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.912602:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f48e93 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d17 expref: 8 pid: 6529 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.912619:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826263.912625:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49029 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49022 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.912635:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826263.912678:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332501376:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 129us (402us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826263.912690:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21814 00010000:00010000:3.0:1787826263.912721:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49029 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49022 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.912735:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49029 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49022 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.912765:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49029 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49022 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826263.912780:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346079872:0@lo:105: 00000100:00100000:2.0:1787826263.912824:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346079488:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787826263.912841:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.912852:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826263.912906:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826263.912944:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.913147:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787826263.913159:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346079872 00000100:00100000:2.0:1787826263.913164:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346079872 00000100:00100000:2.0:1787826263.913197:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7790 00000100:00100000:2.0:1787826263.913204:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c09d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346079872:12345-0@lo:105: 00010000:00010000:2.0:1787826263.913214:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913224:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913234:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913243:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913252:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913262:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913286:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ee600) 00010000:00010000:3.0:1787826263.913298:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.913309:0:11629:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826263.913330:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c09d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346079872:12345-0@lo:105: Request processed in 125us (423us total) trans 0 rc 0/0 00000004:00080000:3.0:1787826263.913335:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826263.913340:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7790 00010000:00010000:3.0:1787826263.913374:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.913382:0:11629:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.913391:0:11629:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826263.913408:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080000:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787826263.913435:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826263.913462:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.913478:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346080000 00000100:00100000:2.0:1787826263.913484:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346080000 00000100:00100000:3.0:1787826263.913488:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346079872:0@lo:105: 00010000:00010000:3.0:1787826263.913513:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826263.913516:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23173 00000100:00100000:2.0:1787826263.913527:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:11629:x1874663346080000:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787826263.913542:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1787826263.913548:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dfd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332501504:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 904us (1206us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826263.913556:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21815 00010000:00010000:2.0:1787826263.913560:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49037 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49030 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913578:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49037 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49030 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913595:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49037 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49030 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.913602:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8b00) 00010000:00010000:2.0:1787826263.913605:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826263.913608:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826263.913644:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:11629:x1874663346080000:12345-0@lo:101:mdt00_004.0 Request processed in 120us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.913651:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23173 00000100:00100000:3.0:1787826263.913671:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080000:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787826263.913683:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.913692:0:11629:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826263.913704:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826263.913737:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080128:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826263.913762:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826263.913785:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.913821:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346080128 00000100:00100000:2.0:1787826263.913826:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346080128 00000100:00100000:2.0:1787826263.913857:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23174 00000100:00100000:2.0:1787826263.913864:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:11629:x1874663346080128:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787826263.913932:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:11629:x1874663346080128:12345-0@lo:1000:mdt00_004.0 Request processed in 69us (169us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.913939:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23174 00000100:00100000:0.0:1787826263.914008:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080128:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787826263.914050:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080256:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787826263.914082:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826263.914162:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.914258:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346080256 00000100:00100000:2.0:1787826263.914263:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346080256 00000100:00100000:2.0:1787826263.914296:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23175 00000100:00100000:2.0:1787826263.914303:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:11629:x1874663346080256:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787826263.914407:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:11629:x1874663346080256:12345-0@lo:1000:mdt00_004.0 Request processed in 104us (323us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826263.914414:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23175 00000100:00100000:3.0:1787826263.915302:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080256:0@lo:1000:mdt00_004.0 00010000:00010000:3.0:1787826263.915370:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.915379:0:11629:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.915389:0:11629:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826263.915416:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080384:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787826263.915452:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826263.915482:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.915932:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346080384 00000100:00100000:2.0:1787826263.915939:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346080384 00000100:00100000:2.0:1787826263.915998:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23176 00000100:00100000:2.0:1787826263.916007:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:11629:x1874663346080384:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787826263.916020:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826263.916036:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4903e expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.916060:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.916069:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4903e expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.916094:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.916135:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 6 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826263.916147:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346080512:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826263.916209:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bcb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826263.926160:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826263.926219:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346080512:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826263.926240:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332501632 00000100:00100000:3.0:1787826263.926252:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332501632 00010000:00010000:1.0:1787826263.926260:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4903e expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.926276:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:1.0:1787826263.926279:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826263.926283:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4903e expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.926294:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00010000:00010000:3.0:1787826263.926326:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 7 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.926371:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:11629:x1874663346080384:12345-0@lo:101:mdt00_004.0 Request processed in 10363us (10916us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.926380:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23176 00000100:00100000:3.0:1787826263.926454:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21816 00000100:00100000:3.0:1787826263.926469:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332501632:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826263.926492:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826263.926502:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 7 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.926515:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 7 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826263.926533:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080384:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787826263.926538:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f48eaf lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d1e expref: 7 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.926554:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:0.0:1787826263.926556:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.926560:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4903e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.926580:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.926582:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4903e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.926602:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4903e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.926794:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4903e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826263.926838:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b758000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346080640:0@lo:105: 00000100:00100000:3.0:1787826263.927071:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826263.927130:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346080640 00000100:00100000:2.0:1787826263.927136:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346080640 00000100:00100000:2.0:1787826263.927172:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7791 00000100:00100000:2.0:1787826263.927180:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346080640:12345-0@lo:105: 00010000:00010000:2.0:1787826263.927191:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.927205:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.927214:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.927222:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.927228:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.927235:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.927256:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68c00) 00000100:00100000:2.0:1787826263.927288:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346080640:12345-0@lo:105: Request processed in 109us (218us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.927294:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7791 00010000:00010000:0.0:1787826263.927354:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.927363:0:11629:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.927408:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.927419:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.927427:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.927445:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.927455:0:11629:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.927462:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826263.927475:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e6c0 going to sleep for 11 seconds 00010000:00010000:0.0:1787826263.927487:0:11629:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826263.927495:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b758000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346080640:0@lo:105: 00010000:00010000:0.0:1787826263.927502:0:11629:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 11 pid: 6527 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.927510:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826263.927523:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:11629:1874663346080768:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826263.927564:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332501632:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1098us (1412us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826263.927568:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826263.927575:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21816 00000100:00100000:0.0:1787826263.934671:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:11629:1874663346080768:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826263.934673:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:0.0:1787826263.934709:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.936187:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332501760 00000100:00100000:1.0:1787826263.936200:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332501760 00010000:00010000:1.0:1787826263.936248:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 12 pid: 6527 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.936296:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21817 00000100:00100000:1.0:1787826263.936306:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332501760:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826263.936320:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826263.936328:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 12 pid: 6527 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.936341:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 12 pid: 6527 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.936364:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f48f81 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d56 expref: 12 pid: 6527 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.936380:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f7967ac8f00) 00010000:00010000:1.0:1787826263.936385:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.936395:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.936403:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.936435:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826263.936468:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.936479:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826263.936496:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332501760:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 191us (1832us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.936504:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21817 00010000:00010000:0.0:1787826263.936530:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.936540:0:11629:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826263.936551:0:11629:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826263.936571:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080896:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787826263.936620:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826263.936664:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.936926:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346080896 00000100:00100000:1.0:1787826263.936932:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346080896 00000100:00100000:1.0:1787826263.936973:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23177 00000100:00100000:1.0:1787826263.936993:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:11629:x1874663346080896:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787826263.937009:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826263.937025:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49061 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4905a expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.937044:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49061 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4905a expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.937057:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49061 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4905a expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.937064:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7967ac1400) 00010000:00010000:1.0:1787826263.937068:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826263.937070:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:3.0:1787826263.937163:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346080896:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787826263.937170:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:11629:x1874663346080896:12345-0@lo:101:mdt00_004.0 Request processed in 178us (548us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.937178:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23177 00010000:00010000:3.0:1787826263.937181:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.937193:0:11629:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826263.937951:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346081024:0@lo:403:mdt00_004.0 00000100:00100000:3.0:1787826263.938007:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826263.938040:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.938097:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346081024 00000100:00100000:1.0:1787826263.938138:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346081024 00000100:00100000:1.0:1787826263.938183:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23178 00000100:00100000:1.0:1787826263.938193:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:11629:x1874663346081024:12345-0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787826263.940951:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:11629:x1874663346081024:12345-0@lo:403:mdt00_004.0 Request processed in 2751us (2934us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.940966:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23178 00000100:00100000:3.0:1787826263.941042:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346081024:0@lo:403:mdt00_004.0 00000040:00080000:3.0:1787826263.942248:0:11629:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1168, 952 off1060640 00000040:00080000:3.0:1787826263.943753:0:11629:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1486, 1032 off1499296 00000004:00080000:3.0:1787826263.943767:0:11629:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7848d5e6c0 with version 6768 00000004:00080000:2.0:1787826263.943878:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6768 rpc_version 6768 00000020:00100000:2.0:1787826263.943918:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826263.943929:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78433ed500 x1874663346081280/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826263.943954:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346081280:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826263.944039:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826263.944079:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bcb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826263.944183:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346081280 00000100:00100000:1.0:1787826263.944191:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346081280 00000100:00100000:1.0:1787826263.944251:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23179 00000100:00100000:1.0:1787826263.944261:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346081280:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826263.944695:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346081280:12345-0@lo:1000:osp_up0-1.0 Request processed in 435us (654us total) trans 8589957291 rc 0/0 00000100:00100000:1.0:1787826263.944703:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23179 00000100:00100000:2.0:1787826263.944808:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346081280:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787826263.944875:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3c:0x0]: returning size 4096 00010000:00010000:3.0:1787826263.944897:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.944907:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.944916:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.944921:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826263.944926:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.944934:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.944948:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18c:0x0].0x0 (ffff9f7967ac8f00) 00010000:00010000:3.0:1787826263.944951:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826263.944956:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.944962:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f49053 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.944997:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945005:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945018:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945043:0:11629:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826263.945050:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18a:0x0] 00010000:00010000:3.0:1787826263.945059:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945065:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945071:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945076:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826263.945079:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945086:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945095:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:3.0:1787826263.945098:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826263.945102:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945132:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826263.945139:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945154:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4904c lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945157:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826263.945164:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945170:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826263.945175:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945184:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945192:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945207:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945209:0:11629:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826263.945218:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000004:00080000:3.0:1787826263.945221:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826263.945228:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945236:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945243:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945260:0:11629:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787826263.945265:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826263.945271:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346081408:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826263.945273:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826263.945280:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945287:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826263.945291:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826263.945302:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826263.945312:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826263.945329:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.945336:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826263.945367:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 1/0,0 mode: --/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945381:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4905a lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49061 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945401:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826263.945404:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945409:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826263.945413:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826263.945420:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945439:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945445:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826263.945458:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945464:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49022 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49029 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945473:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826263.945474:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826263.945476:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826263.945484:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346081664:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826263.945509:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826263.945558:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826263.945564:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346081536:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826263.945584:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826263.945620:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346081408 00000100:00100000:2.0:1787826263.945625:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346081408 00000100:00100000:2.0:1787826263.945657:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346081664 00000100:00100000:2.0:1787826263.945659:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346081664 00000100:00100000:2.0:1787826263.945672:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346081536 00000100:00100000:2.0:1787826263.945674:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346081536 00000100:00100000:2.0:1787826263.945687:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10965 00000100:00100000:2.0:1787826263.945694:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3652:x1874663346081408:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826263.945705:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826263.945719:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49061 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4905a expref: 16 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.945731:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x0 (ffff9f7967ac1400) 00010000:00010000:2.0:1787826263.945737:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826263.945814:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826263.945854:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10966 00000100:00100000:1.0:1787826263.945864:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+15:3654:x1874663346081664:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826263.945873:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826263.945885:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49029 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49022 expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.945897:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826263.945900:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826263.945939:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+14:3654:x1874663346081664:12345-0@lo:103:ldlm_bl.0 Request processed in 77us (428us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.945945:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10966 00000100:00100000:1.0:1787826263.945963:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10967 00000100:00100000:1.0:1787826263.945968:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346081536:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826263.945974:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826263.945996:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49045 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4903e expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826263.946006:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:1.0:1787826263.946012:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826263.946036:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346081536:12345-0@lo:103:ldlm_bl.0 Request processed in 68us (450us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826263.946041:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10967 00000100:00100000:0.0:1787826263.946144:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346081664:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826263.946231:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346081408:12345-0@lo:103:ldlm_bl.0 Request processed in 536us (922us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826263.946237:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10965 00000100:00100000:2.0:1787826263.946268:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346081536:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826263.946279:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346081408:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826263.947332:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826263.947354:0:11629:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826263.947508:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d814a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223172:x1874663332501248:12345-192.168.203.56@tcp:36:rm.0 Request processed in 40620us (40722us total) trans 4294985309 rc 0/0 00000100:00100000:2.0:1787826263.947517:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68156 00010000:00010000:3.0:1787826263.948073:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.948085:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4903e lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49045 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826263.948140:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826263.948143:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826263.948146:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826264.027712:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826264.027858:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332501888 00000100:00100000:2.0:1787826264.027868:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332501888 00000100:00100000:2.0:1787826264.027942:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68157 00000100:00100000:2.0:1787826264.027952:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223180:x1874663332501888:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826264.028013:0:24702:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345594368 00000100:00100000:2.0:1787826264.028085:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223180:x1874663332501888:12345-192.168.203.56@tcp:41:stat.0 Request processed in 134us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.028092:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68157 00000100:00080000:0.0:1787826264.028201:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826264.047157:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826264.047186:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346081792:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.047283:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826264.047345:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346082176:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.047381:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826264.047401:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346082688:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.047422:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826264.047437:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1787826264.047442:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346082304:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.047459:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826264.047471:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346082816:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.047496:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826264.047524:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346082176 00000100:00100000:0.0:1787826264.047532:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346082176 00000100:00100000:0.0:1787826264.047584:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14205 00000100:00100000:0.0:1787826264.047594:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346082176:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.047645:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346082176:12345-0@lo:400:kworker.0 Request processed in 52us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826264.047652:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14205 00000100:00080000:0.0:1787826264.047678:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826264.047705:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346082176:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.047844:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346082688 00000100:00100000:3.0:1787826264.047849:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346082688 00000100:00100000:3.0:1787826264.047891:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346082304 00000100:00100000:3.0:1787826264.047894:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346082304 00000100:00100000:3.0:1787826264.047913:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346082816 00000100:00100000:3.0:1787826264.047916:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346082816 00000100:00100000:3.0:1787826264.047934:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8746 00000100:00100000:3.0:1787826264.047940:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346082688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.047988:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346082688:12345-0@lo:400:kworker.0 Request processed in 37us (553us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.047994:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8746 00000100:00100000:3.0:1787826264.048010:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8747 00000100:00100000:3.0:1787826264.048016:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346082304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048043:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346082304:12345-0@lo:400:kworker.0 Request processed in 28us (583us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.048048:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8747 00000100:00100000:3.0:1787826264.048059:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8748 00000100:00100000:3.0:1787826264.048064:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346082816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048090:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346082816:12345-0@lo:400:kworker.0 Request processed in 26us (602us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.048096:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8748 00000100:00100000:3.0:1787826264.048148:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346081920:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048175:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.048193:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346082432:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048212:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.048226:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346082944:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048244:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.048262:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1787826264.048267:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346082048:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048290:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.048306:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967911f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346082560:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048326:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.048349:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346081920 00000100:00100000:3.0:1787826264.048353:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346081920 00000100:00100000:3.0:1787826264.048377:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346082432 00000100:00100000:3.0:1787826264.048380:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346082432 00000100:00100000:3.0:1787826264.048397:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346082944 00000100:00100000:3.0:1787826264.048400:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346082944 00000100:00100000:3.0:1787826264.048413:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346082048 00000100:00100000:3.0:1787826264.048416:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346082048 00000100:00100000:3.0:1787826264.048436:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346082560 00000100:00100000:3.0:1787826264.048440:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346082560 00000100:00100000:3.0:1787826264.048459:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14206 00000100:00100000:3.0:1787826264.048465:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346081920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048507:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346081920:12345-0@lo:400:kworker.0 Request processed in 42us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.048513:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14206 00000100:00100000:3.0:1787826264.048527:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14207 00000100:00100000:3.0:1787826264.048534:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346082432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048561:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346082432:12345-0@lo:400:kworker.0 Request processed in 28us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.048567:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14207 00000100:00100000:3.0:1787826264.048579:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14208 00000100:00100000:3.0:1787826264.048585:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346082944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048610:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346082944:12345-0@lo:400:kworker.0 Request processed in 25us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.048615:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14208 00000100:00100000:3.0:1787826264.048627:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14209 00000100:00100000:3.0:1787826264.048633:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346082048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048654:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346082048:12345-0@lo:400:kworker.0 Request processed in 22us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.048660:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14209 00000100:00100000:3.0:1787826264.048671:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14210 00000100:00100000:3.0:1787826264.048678:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346082560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048700:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346082560:12345-0@lo:400:kworker.0 Request processed in 23us (374us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.048706:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14210 00000100:00100000:3.0:1787826264.048751:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346082048:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048761:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967911f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346082560:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048770:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346081920:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048778:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346082432:0@lo:400:kworker.0 00000100:00100000:3.0:1787826264.048786:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346082944:0@lo:400:kworker.0 00000100:00100000:1.0:1787826264.049137:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346081792 00000100:00100000:1.0:1787826264.049143:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346081792 00000100:00100000:1.0:1787826264.049187:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2203 00000100:00100000:1.0:1787826264.049195:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847253050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346081792:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826264.049237:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847253050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346081792:12345-0@lo:400:kworker.0 Request processed in 43us (1954us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826264.049244:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2203 00000100:00080000:0.0:1787826264.049590:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787826264.049596:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787826264.049599:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787826264.049602:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826264.049633:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346082304:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.049646:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346082816:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.049656:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346082688:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.049665:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346081792:0@lo:400:kworker.0 00000100:00100000:0.0:1787826264.133395:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826264.133504:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332502016 00000100:00100000:2.0:1787826264.133515:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332502016 00000100:00100000:2.0:1787826264.133578:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68158 00000100:00100000:2.0:1787826264.133589:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223184:x1874663332502016:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826264.133610:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826264.133631:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49068 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d80 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.133700:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.133715:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.133726:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826264.133771:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826264.133774:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826264.133793:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.133817:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49068 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.133835:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.133844:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826264.133847:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826264.133850:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826264.133915:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223184:x1874663332502016:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 327us (529us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.133922:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68158 00000100:00100000:1.0:1787826264.139837:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826264.139935:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332502144 00000100:00100000:0.0:1787826264.139946:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332502144 00000100:00100000:0.0:1787826264.140021:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68159 00000100:00100000:0.0:1787826264.140032:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223184:x1874663332502144:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826264.140054:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826264.140085:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f49076 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d87 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.140179:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.140194:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.140205:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.140216:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.140225:0:24703:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49037 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49030 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.140234:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.140269:0:24703:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49037 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49030 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826264.140286:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346083072:0@lo:104: 00000100:00100000:0.0:1787826264.140348:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826264.140388:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d8a6e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826264.140516:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346083072 00000100:00100000:1.0:1787826264.140521:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346083072 00000100:00100000:1.0:1787826264.140562:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7792 00000100:00100000:1.0:1787826264.140571:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346083072:12345-0@lo:104: 00010000:00010000:1.0:1787826264.140584:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.140641:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826264.140647:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346083072:12345-0@lo:104: Request processed in 77us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826264.140653:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7792 00010000:00010000:0.0:1787826264.141147:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826264.141155:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.141163:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826264.141169:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826264.141178:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.141207:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.141214:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826264.141244:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.141252:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49030 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49037 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.141270:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826264.141273:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826264.141275:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826264.141301:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a300 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346083072:0@lo:104: 00010000:00010000:0.0:1787826264.141324:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826264.141443:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826264.141458:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346083200:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826264.141506:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826264.141630:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346083200 00000100:00100000:1.0:1787826264.141636:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346083200 00010000:00010000:1.0:1787826264.141658:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49037 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49030 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826264.141696:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21818 00000100:00100000:1.0:1787826264.141705:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346083200:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826264.141715:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787826264.147573:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957291 is committed 00010000:00010000:1.0:1787826264.158868:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49037 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49030 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.158891:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8b00) 00010000:00010000:1.0:1787826264.158897:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.158907:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.158915:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.158954:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826264.159049:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346083200:12345-0@lo:103:ldlm_bl.0 Request processed in 17343us (17541us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826264.159069:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21818 00010000:00010000:0.0:1787826264.159189:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159198:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159275:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159283:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159289:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159296:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159302:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826264.159307:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159314:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159325:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826264.159328:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826264.159345:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159352:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4907d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159362:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159368:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159373:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159378:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826264.159381:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159387:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159394:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8b00) 00010000:00010000:0.0:1787826264.159397:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826264.159400:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159407:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49084 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159448:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f49076 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d87 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159459:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826264.159461:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826264.159465:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f49076 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d87 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.159473:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826264.159557:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223184:x1874663332502144:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 19526us (19730us total) trans 0 rc 301/301 00000100:00100000:0.0:1787826264.159564:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68159 00000100:00100000:3.0:1787826264.160586:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346083200:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826264.183485:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826264.184022:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332502272 00000100:00100000:3.0:1787826264.184033:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332502272 00000100:00100000:3.0:1787826264.184140:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68160 00000100:00100000:3.0:1787826264.184153:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223184:x1874663332502272:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826264.184174:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826264.184194:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4908b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d8e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.184270:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.184284:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.184296:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826264.184344:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826264.184348:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826264.184367:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.184392:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4908b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.184411:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.184419:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826264.184422:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826264.184425:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826264.184495:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223184:x1874663332502272:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 344us (1019us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.184503:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68160 00000100:00100000:2.0:1787826264.191548:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826264.191630:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332502528 00000100:00100000:3.0:1787826264.191651:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332502528 00000100:00100000:3.0:1787826264.191736:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68161 00000100:00100000:3.0:1787826264.191747:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223184:x1874663332502528:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:3.0:1787826264.191884:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346083328:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.191935:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.191968:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826264.192094:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346083328 00000100:00100000:2.0:1787826264.192099:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346083328 00000100:00100000:2.0:1787826264.192159:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23180 00000100:00100000:2.0:1787826264.192166:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346083328:12345-0@lo:1000:mdt00_009.0 00000100:00100000:2.0:1787826264.192258:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346083328:12345-0@lo:1000:mdt00_009.0 Request processed in 92us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.192265:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23180 00000100:00080000:0.0:1787826264.192598:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826264.192668:0:35092:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957291 gen 2 00000004:00080000:3.0:1787826264.192674:0:35092:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957290 committed transno 8589957291 00000004:00080000:3.0:1787826264.192700:0:35092:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957291 committed transno 8589957291 00000100:00100000:3.0:1787826264.192718:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346083328:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.192763:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346083456:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.192792:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.192818:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826264.192871:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346083456 00000100:00100000:1.0:1787826264.192876:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346083456 00000100:00100000:1.0:1787826264.192913:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23181 00000100:00100000:1.0:1787826264.192921:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346083456:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826264.193028:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346083456:12345-0@lo:1000:mdt00_009.0 Request processed in 107us (234us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826264.193035:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23181 00000100:00100000:3.0:1787826264.193569:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346083456:0@lo:1000:mdt00_009.0 00010000:00010000:3.0:1787826264.193620:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.193630:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.193639:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826264.193652:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346083584:0@lo:101:mdt00_009.0 00000100:00100000:3.0:1787826264.193681:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.193709:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f796ed379c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826264.193768:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346083584 00000100:00100000:2.0:1787826264.193773:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346083584 00000100:00100000:2.0:1787826264.193811:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23182 00000100:00100000:2.0:1787826264.193819:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346083584:12345-0@lo:101:mdt00_009.0 00010000:00010000:2.0:1787826264.193831:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826264.193846:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49099 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.193871:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.193881:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.193890:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49099 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.193914:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.193926:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 7 pid: 35094 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826264.193936:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305c00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346083712:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826264.194016:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.194026:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 6 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826264.194035:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346083840:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826264.194069:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851d70600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826264.197854:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826264.197899:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826264.198006:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332502656 00000100:00100000:0.0:1787826264.198016:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332502656 00010000:00010000:0.0:1787826264.198069:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 8 pid: 35094 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826264.198133:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332502784 00000100:00100000:0.0:1787826264.198137:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332502784 00010000:00010000:0.0:1787826264.198152:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 7 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826264.198173:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21819 00000100:00100000:0.0:1787826264.198183:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c82450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332502656:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826264.198196:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826264.198202:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 8 pid: 35094 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.198222:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 8 pid: 35094 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.198244:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00000100:00100000:1.0:1787826264.198350:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21820 00000100:00100000:1.0:1787826264.198361:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332502784:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826264.198371:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826264.198377:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 7 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.200079:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49099 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.200097:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826264.200183:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c82450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332502656:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1999us (2337us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826264.200194:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21819 00010000:00010000:2.0:1787826264.201949:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49092 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d8e expref: 7 pid: 35094 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826264.201973:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305c00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346083712:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826264.201998:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346083840:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826264.207355:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 7 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.207390:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4906f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800d80 expref: 7 pid: 24702 timeout: 8176 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.207411:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826264.207417:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49099 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.207428:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49099 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.207436:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49099 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.207483:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49099 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826264.207499:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346083968:0@lo:105: 00000100:00100000:1.0:1787826264.207562:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826264.207609:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02df00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826264.207697:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346083968 00000100:00100000:0.0:1787826264.207705:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346083968 00000100:00100000:0.0:1787826264.207761:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7793 00000100:00100000:0.0:1787826264.207771:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346083968:12345-0@lo:105: 00010000:00010000:0.0:1787826264.207782:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.207792:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.207800:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.207808:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.207819:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.207827:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.207843:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68c00) 00000100:00100000:0.0:1787826264.207898:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346083968:12345-0@lo:105: Request processed in 129us (335us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826264.207905:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7793 00000100:00100000:3.0:1787826264.208005:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346083968:0@lo:105: 00010000:00010000:3.0:1787826264.208025:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826264.208084:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332502784:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 9726us (10183us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.208092:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21820 00010000:00010000:2.0:1787826264.213502:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49099 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.213520:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826264.213522:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826264.213526:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826264.213622:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35092:x1874663346083584:12345-0@lo:101:mdt00_009.0 Request processed in 19802us (19937us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.213630:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23182 00000100:00100000:3.0:1787826264.213841:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d814e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346083584:0@lo:101:mdt00_009.0 00010000:00010000:3.0:1787826264.213863:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826264.213890:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826264.213934:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.213942:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.213951:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826264.213971:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967911f80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084096:0@lo:101:mdt00_009.0 00000100:00100000:3.0:1787826264.214016:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.214047:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f796ed379c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826264.214483:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346084096 00000100:00100000:1.0:1787826264.214491:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346084096 00000100:00100000:1.0:1787826264.214542:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23183 00000100:00100000:1.0:1787826264.214550:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:35092:x1874663346084096:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826264.214568:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826264.214586:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f490ae lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f490a7 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.214611:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f490ae lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f490a7 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.214624:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f490ae lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f490a7 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.214631:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1300) 00010000:00010000:1.0:1787826264.214634:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826264.214637:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826264.214683:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084096:12345-0@lo:101:mdt00_009.0 Request processed in 133us (666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826264.214689:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23183 00000100:00100000:3.0:1787826264.214835:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967911f80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084096:0@lo:101:mdt00_009.0 00010000:00010000:3.0:1787826264.214848:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.214859:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826264.214878:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826264.214913:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084224:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.214941:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.214963:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826264.215022:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346084224 00000100:00100000:1.0:1787826264.215027:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346084224 00000100:00100000:1.0:1787826264.215067:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23184 00000100:00100000:1.0:1787826264.215074:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084224:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826264.215161:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084224:12345-0@lo:1000:mdt00_009.0 Request processed in 87us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826264.215167:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23184 00000100:00100000:3.0:1787826264.215354:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084224:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.215394:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084352:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.215421:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.215443:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826264.215540:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346084352 00000100:00100000:2.0:1787826264.215545:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346084352 00000100:00100000:2.0:1787826264.215579:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23185 00000100:00100000:2.0:1787826264.215586:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084352:12345-0@lo:1000:mdt00_009.0 00000100:00100000:2.0:1787826264.215677:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084352:12345-0@lo:1000:mdt00_009.0 Request processed in 92us (255us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826264.215685:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23185 00000100:00100000:3.0:1787826264.215790:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084352:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.215862:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084480:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.215885:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.215912:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826264.215962:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346084480 00000100:00100000:0.0:1787826264.215967:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346084480 00000100:00100000:0.0:1787826264.216013:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23186 00000100:00100000:0.0:1787826264.216021:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084480:12345-0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826264.216096:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084480:12345-0@lo:1000:mdt00_009.0 Request processed in 76us (209us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826264.216102:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23186 00000100:00100000:3.0:1787826264.216193:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084480:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.216242:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084608:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.216276:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.216297:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826264.216450:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346084608 00000100:00100000:0.0:1787826264.216455:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346084608 00000100:00100000:0.0:1787826264.216485:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23187 00000100:00100000:0.0:1787826264.216492:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084608:12345-0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826264.216538:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084608:12345-0@lo:1000:mdt00_009.0 Request processed in 47us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826264.216545:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23187 00000100:00100000:3.0:1787826264.216696:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084608:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.216735:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084736:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.216756:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.216777:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826264.216789:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346084736 00000100:00100000:3.0:1787826264.216792:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346084736 00000100:00100000:3.0:1787826264.216820:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23188 00000100:00100000:3.0:1787826264.216827:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084736:12345-0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.216891:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084736:12345-0@lo:1000:mdt00_009.0 Request processed in 64us (134us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.216897:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23188 00000100:00100000:3.0:1787826264.216921:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084736:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.216956:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084864:0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.216988:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.217002:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d58840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826264.217013:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346084864 00000100:00100000:3.0:1787826264.217016:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346084864 00000100:00100000:3.0:1787826264.217032:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23189 00000100:00100000:3.0:1787826264.217037:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084864:12345-0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826264.217091:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346084864:12345-0@lo:1000:mdt00_009.0 Request processed in 52us (102us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.217096:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23189 00000100:00100000:3.0:1787826264.217142:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346084864:0@lo:1000:mdt00_009.0 00000040:00080000:3.0:1787826264.218815:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1169, 2168 off1062808 00000040:00080000:3.0:1787826264.218903:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1487, 2368 off1501664 00000004:00080000:3.0:1787826264.218909:0:35092:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7848d58000 with version 6769 00000004:00080000:2.0:1787826264.225512:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6769 rpc_version 6769 00000020:00100000:2.0:1787826264.225575:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826264.225587:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b304700 x1874663346085120/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826264.225613:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346085120:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826264.225688:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826264.225733:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851d70600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826264.226289:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346085120 00000100:00100000:3.0:1787826264.226297:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346085120 00000100:00100000:3.0:1787826264.226472:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23190 00000100:00100000:3.0:1787826264.226482:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346085120:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826264.228491:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346085120:12345-0@lo:1000:osp_up0-1.0 Request processed in 2001us (2793us total) trans 8589957292 rc 0/0 00000100:00100000:3.0:1787826264.228506:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23190 00000100:00100000:2.0:1787826264.235711:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346085120:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787826264.236066:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.236083:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.236094:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.236142:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.236150:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.236156:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.236189:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.236195:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.236204:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826264.236225:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346085248:0@lo:101:mdt00_009.0 00000100:00100000:3.0:1787826264.236359:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.236389:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851ff5840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826264.238021:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346085248 00000100:00100000:2.0:1787826264.238034:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346085248 00000100:00100000:2.0:1787826264.238101:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23191 00000100:00100000:2.0:1787826264.238137:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:35092:x1874663346085248:12345-0@lo:101:mdt00_009.0 00010000:00010000:2.0:1787826264.238162:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826264.238185:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f490ca lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f490c3 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.238214:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f490ca lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f490c3 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.238228:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f490ca lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f490c3 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.238236:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:2.0:1787826264.238239:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826264.238441:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826264.238520:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35092:x1874663346085248:12345-0@lo:101:mdt00_009.0 Request processed in 385us (2161us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.238527:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23191 00000100:00100000:3.0:1787826264.238690:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346085248:0@lo:101:mdt00_009.0 00010000:00010000:3.0:1787826264.238708:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238719:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826264.238746:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000007012922 reply state 00000000d9ae6efc transno 4294985310 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238758:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238765:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7961d81000) 00010000:00010000:3.0:1787826264.238767:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826264.238770:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826264.238776:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238785:0:35092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826264.238797:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00000004:00010000:3.0:1787826264.238805:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000007012922 reply state 00000000d9ae6efc transno 4294985310 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238813:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238818:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81b00) 00010000:00010000:3.0:1787826264.238820:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826264.238823:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787826264.238828:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3e:0x0]: returning size 4096 00000004:00080000:3.0:1787826264.238837:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826264.238847:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238854:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238861:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238964:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826264.238970:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826264.238984:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.238992:0:35092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826264.239075:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223184:x1874663332502528:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 47329us (47536us total) trans 4294985310 rc 0/0 00000100:00100000:3.0:1787826264.239083:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68161 00010000:00010000:0.0:1787826264.246818:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826264.246834:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.246846:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826264.246855:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826264.246864:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.246916:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.246923:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826264.247019:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:0.0:1787826264.247102:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.247133:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f49099 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f490a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.247151:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826264.247152:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826264.247154:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826264.247413:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826264.255910:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346085376:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826264.256005:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826264.256158:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826264.256241:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346085376 00000100:00100000:0.0:1787826264.256259:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346085376 00000100:00100000:0.0:1787826264.256311:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10968 00000100:00100000:0.0:1787826264.256321:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346085376:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826264.256347:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826264.256372:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f490a0 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49099 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.256394:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826264.256398:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826264.256443:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346085376:12345-0@lo:103:ldlm_bl.0 Request processed in 124us (439us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826264.256450:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10968 00000100:00100000:2.0:1787826264.256588:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346085376:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826264.257363:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332502912 00000100:00100000:3.0:1787826264.257371:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332502912 00000100:00100000:3.0:1787826264.257424:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68162 00000100:00100000:3.0:1787826264.257435:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223173:x1874663332502912:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826264.257453:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826264.257472:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f490d1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d95 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.257535:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f490d8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.257550:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f490d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.257562:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f490d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826264.257603:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826264.257607:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826264.257627:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f490d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.257649:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f490d1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d95 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.257665:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f490d8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d95 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.257671:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826264.257674:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826264.257677:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68800, rc 0) 00000100:00100000:3.0:1787826264.257783:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223173:x1874663332502912:12345-192.168.203.56@tcp:101:bash.0 Request processed in 350us (1626us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.257790:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68162 00000100:00100000:2.0:1787826264.265557:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826264.265646:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332503040 00000100:00100000:1.0:1787826264.265657:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332503040 00000100:00100000:1.0:1787826264.265719:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68163 00000100:00100000:1.0:1787826264.265731:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223173:x1874663332503040:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826264.265753:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826264.265775:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f490df lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800d9c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.265844:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.265859:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.265870:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.265882:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.265892:0:35080:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f490ae lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f490a7 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.265900:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.265933:0:35080:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f490ae lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f490a7 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826264.265949:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000_UUID:35080:1874663346085504:0@lo:104: 00000100:00100000:1.0:1787826264.266007:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826264.266049:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02df00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826264.266130:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346085504 00000100:00100000:3.0:1787826264.266135:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346085504 00000100:00100000:3.0:1787826264.266172:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7794 00000100:00100000:3.0:1787826264.266179:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35080:x1874663346085504:12345-0@lo:104: 00010000:00010000:3.0:1787826264.266191:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.266258:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826264.266265:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35080:x1874663346085504:12345-0@lo:104: Request processed in 86us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.266272:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7794 00010000:00010000:3.0:1787826264.266292:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826264.266298:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.266305:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826264.266311:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826264.266320:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.266361:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.266375:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826264.266392:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.266399:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f490a7 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f490ae expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.266417:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826264.266419:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826264.266421:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826264.266756:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346085632:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826264.266792:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826264.266852:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000_UUID:35080:1874663346085504:0@lo:104: 00010000:00010000:2.0:1787826264.266872:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826264.267194:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346085632 00000100:00100000:3.0:1787826264.267200:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346085632 00010000:00010000:3.0:1787826264.267225:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f490ae lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f490a7 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826264.267361:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21821 00000100:00100000:3.0:1787826264.267370:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346085632:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826264.267382:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787826264.278491:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957292 is committed 00010000:00010000:2.0:1787826264.278628:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f490ae lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f490a7 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.278649:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1300) 00010000:00010000:2.0:1787826264.278654:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.278664:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.278670:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.278709:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826264.278804:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346085632:12345-0@lo:103:ldlm_bl.0 Request processed in 11434us (12010us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.278819:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21821 00000100:00100000:3.0:1787826264.278954:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346085632:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826264.279101:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.279154:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826264.279294:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-mdtlov_UUID:35080:1874663346085760:0@lo:1000:mdt00_008.0 00000100:00100000:1.0:1787826264.279351:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826264.279372:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826264.279553:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346085760 00000100:00100000:2.0:1787826264.279561:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346085760 00000100:00100000:2.0:1787826264.279609:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23192 00000100:00100000:2.0:1787826264.279618:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:35080:x1874663346085760:12345-0@lo:1000:mdt00_008.0 00000100:00100000:2.0:1787826264.279700:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:35080:x1874663346085760:12345-0@lo:1000:mdt00_008.0 Request processed in 83us (349us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.279706:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23192 00000100:00100000:1.0:1787826264.279882:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-mdtlov_UUID:35080:1874663346085760:0@lo:1000:mdt00_008.0 00010000:00010000:1.0:1787826264.279930:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f490f4 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.279943:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f490f4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.279954:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f490f4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826264.279985:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f490f4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.279999:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280006:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280013:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280019:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280029:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826264.280033:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280040:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280052:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826264.280055:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826264.280060:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280065:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f490e6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280077:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280082:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280088:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280093:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826264.280096:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280391:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280407:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1300) 00010000:00010000:1.0:1787826264.280409:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826264.280413:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280419:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f490ed lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280459:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f490df lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d9c expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280473:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f490f4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d9c expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.280481:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7845cb1c00) 00010000:00010000:1.0:1787826264.280483:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826264.280486:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826264.280597:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223173:x1874663332503040:12345-192.168.203.56@tcp:101:bash.0 Request processed in 14867us (15048us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826264.280605:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68163 00000100:00100000:3.0:1787826264.300613:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826264.300719:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332503168 00000100:00100000:2.0:1787826264.300729:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332503168 00000100:00100000:2.0:1787826264.300797:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68164 00000100:00100000:2.0:1787826264.300809:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223173:x1874663332503168:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826264.300830:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826264.300851:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f490fb lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800da3 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.300913:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49102 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.300927:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49102 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.300940:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49102 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826264.301056:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3e:0x0]: returning size 4096 00000004:00200000:2.0:1787826264.301061:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826264.301101:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f490fb lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800da3 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.301164:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49102 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800da3 expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.301173:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81b00) 00010000:00010000:2.0:1787826264.301176:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826264.301179:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826264.301234:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223173:x1874663332503168:12345-192.168.203.56@tcp:101:bash.0 Request processed in 427us (632us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.301241:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68164 00000100:00100000:2.0:1787826264.304720:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826264.304803:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332503296 00000100:00100000:3.0:1787826264.304815:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332503296 00000100:00100000:3.0:1787826264.304876:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68165 00000100:00100000:3.0:1787826264.304887:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223173:x1874663332503296:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826264.304907:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826264.304927:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49109 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800daa expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.304997:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49110 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.305012:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49110 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.305023:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49110 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826264.305076:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18d:0x0]: returning size 4096 00000004:00200000:3.0:1787826264.305081:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826264.305163:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49109 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800daa expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.305187:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49110 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800daa expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.305196:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7961d81000) 00010000:00010000:3.0:1787826264.305200:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826264.305203:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826264.305257:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223173:x1874663332503296:12345-192.168.203.56@tcp:101:bash.0 Request processed in 373us (548us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.305264:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68165 00000100:00100000:1.0:1787826264.308439:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826264.308562:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332503424 00000100:00100000:2.0:1787826264.308572:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332503424 00000100:00100000:2.0:1787826264.308630:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68166 00000100:00100000:2.0:1787826264.308642:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223173:x1874663332503424:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826264.308663:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826264.308684:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49117 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800db1 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.308743:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.308757:0:35095:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f490ca lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f490c3 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.308766:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 4/1,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.308805:0:35095:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f490ca lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f490c3 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826264.308820:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346085888:0@lo:104: 00000100:00100000:2.0:1787826264.308866:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826264.308911:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826264.309074:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346085888 00000100:00100000:1.0:1787826264.309079:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346085888 00000100:00100000:1.0:1787826264.309142:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7795 00000100:00100000:1.0:1787826264.309150:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346085888:12345-0@lo:104: 00010000:00010000:1.0:1787826264.309162:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.309220:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826264.309226:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346085888:12345-0@lo:104: Request processed in 77us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826264.309233:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7795 00010000:00010000:0.0:1787826264.309293:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826264.309300:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.309307:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826264.309313:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826264.309321:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.309361:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.309369:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826264.309402:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.309409:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f490c3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f490ca expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.309429:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826264.309431:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826264.309433:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826264.309441:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826264.309450:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346086016:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826264.309485:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826264.309515:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346086016 00000100:00100000:0.0:1787826264.309518:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346086016 00010000:00010000:0.0:1787826264.309538:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f490ca lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f490c3 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826264.309558:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21822 00000100:00100000:0.0:1787826264.309565:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346086016:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826264.309574:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787826264.310005:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346085888:0@lo:104: 00010000:00010000:2.0:1787826264.310029:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.310413:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f490ca lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f490c3 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.310430:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:0.0:1787826264.310434:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.310443:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.310450:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.310482:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826264.310614:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346086016:12345-0@lo:103:ldlm_bl.0 Request processed in 1048us (1127us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826264.310623:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21822 00000100:00100000:1.0:1787826264.310780:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346086016:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826264.310881:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.310889:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826264.310941:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18b:0x0]: returning size 4096 00000004:00200000:2.0:1787826264.310946:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826264.310996:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49117 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800db1 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.311012:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800db1 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.311019:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:2.0:1787826264.311022:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826264.311024:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826264.311077:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223173:x1874663332503424:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2437us (2648us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.311084:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68166 00000100:00100000:0.0:1787826264.316747:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826264.317227:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332503680 00000100:00100000:3.0:1787826264.317238:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332503680 00000100:00100000:3.0:1787826264.317300:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68167 00000100:00100000:3.0:1787826264.317311:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223173:x1874663332503680:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826264.317330:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826264.317343:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826264.317374:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49110 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800daa expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.317395:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7961d81000) 00010000:00010000:3.0:1787826264.317400:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826264.317413:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49125 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800db8 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.317549:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.317567:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.317575:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.317589:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.317597:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.317604:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826264.317729:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91880 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346086144:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826264.317791:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826264.317831:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851ff5900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826264.317952:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346086144 00000100:00100000:1.0:1787826264.317957:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346086144 00000100:00100000:1.0:1787826264.318008:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23193 00000100:00100000:1.0:1787826264.318017:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35094:x1874663346086144:12345-0@lo:1000:mdt00_011.0 00000100:00100000:1.0:1787826264.318100:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35094:x1874663346086144:12345-0@lo:1000:mdt00_011.0 Request processed in 84us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826264.318142:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23193 00000100:00100000:3.0:1787826264.319489:0:35094:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957292 gen 2 00000004:00080000:3.0:1787826264.319498:0:35094:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957292 committed transno 8589957292 00000100:00100000:3.0:1787826264.319534:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91880 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346086144:0@lo:1000:mdt00_011.0 00000004:00080000:3.0:1787826264.319803:0:35094:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4c9:0x0], index 0: 0 00010000:00010000:3.0:1787826264.319967:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4913a lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.319994:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4913a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320005:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4913a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320043:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320050:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320057:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320064:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320070:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826264.320075:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320082:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320094:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7961d81000) 00010000:00010000:3.0:1787826264.320097:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826264.320101:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320128:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4912c lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826264.320143:0:35094:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000422a6979 reply state 000000008753b013 transno 4294985311 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320153:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320160:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f7961d81400) 00010000:00010000:3.0:1787826264.320162:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826264.320165:0:35094:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826264.320200:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49125 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800db8 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320216:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4913a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800db8 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.320224:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:3.0:1787826264.320226:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826264.320229:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826264.320556:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223173:x1874663332503680:12345-192.168.203.56@tcp:101:bash.0 Request processed in 3245us (3818us total) trans 4294985311 rc 0/0 00000100:00100000:3.0:1787826264.320564:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68167 00000100:00100000:3.0:1787826264.328208:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826264.328293:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332503808 00000100:00100000:2.0:1787826264.328302:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332503808 00000100:00100000:2.0:1787826264.328395:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30765 00000100:00100000:2.0:1787826264.328406:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:223173:x1874663332503808:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826264.328424:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826264.328458:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800dbf expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826264.328483:0:8445:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4c9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826264.328506:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=8076s v=(0 0) 00010000:00010000:2.0:1787826264.328511:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826264.328521:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.328538:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826264.328646:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787826264.329008:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826264.329074:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223173:x1874663332503808:12345-192.168.203.56@tcp:101:bash.0 Request processed in 669us (874us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.329082:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30765 00000100:00100000:1.0:1787826264.336328:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826264.337022:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332504064 00000100:00100000:0.0:1787826264.337034:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332504064 00000100:00100000:0.0:1787826264.337099:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9289 00000100:00100000:0.0:1787826264.337152:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943006850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223173:x1874663332504064:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826264.337442:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943006850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:223173:x1874663332504064:12345-192.168.203.56@tcp:35:bash.0 Request processed in 292us (1123us total) trans 4294985312 rc 0/0 00000100:00100000:0.0:1787826264.337450:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9289 00000100:00100000:0.0:1787826264.340343:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826264.341267:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332504320 00000100:00100000:1.0:1787826264.341279:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332504320 00000100:00100000:1.0:1787826264.341362:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68168 00000100:00100000:1.0:1787826264.341373:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfcd850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223173:x1874663332504320:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826264.341394:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826264.341397:0:35098:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787826264.341425:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4911e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800db1 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.341445:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:1.0:1787826264.341457:0:35098:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826264.341473:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4914f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800dc6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.341618:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.341631:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.341640:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.341653:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.341660:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.341667:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826264.342003:0:35098:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x51c:0x0], index 1: 0 00010000:00010000:1.0:1787826264.342216:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49164 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342226:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49164 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342234:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49164 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342265:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342276:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342282:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342289:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342295:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826264.342300:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342308:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342318:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:1.0:1787826264.342321:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826264.342325:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342331:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49156 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826264.342343:0:35098:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f8de95d4 reply state 00000000e50ab00d transno 8589957293 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342353:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342358:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x4ec94fd6 (ffff9f7845cb1100) 00010000:00010000:1.0:1787826264.342361:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826264.342363:0:35098:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826264.342392:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4914f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dc6 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342408:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49164 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dc6 expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826264.342416:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:1.0:1787826264.342418:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826264.342420:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826264.342483:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfcd850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223173:x1874663332504320:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1112us (2148us total) trans 8589957293 rc 0/0 00000100:00100000:1.0:1787826264.342491:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68168 00000100:00100000:3.0:1787826264.349469:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826264.350212:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332504448 00000100:00100000:2.0:1787826264.350224:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332504448 00000100:00100000:2.0:1787826264.350284:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30766 00000100:00100000:2.0:1787826264.350294:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223173:x1874663332504448:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826264.350314:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826264.350337:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800dcd expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826264.350375:0:8759:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x51c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826264.350399:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51c:0x0].0x0 since=8076s v=(0 0) 00010000:00010000:2.0:1787826264.350403:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826264.350413:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 7 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.350429:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 7 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826264.350443:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826264.350502:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223173:x1874663332504448:12345-192.168.203.56@tcp:101:bash.0 Request processed in 210us (1044us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.350510:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30766 00000100:00100000:0.0:1787826264.354511:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826264.354656:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332504704 00000100:00100000:1.0:1787826264.354666:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332504704 00000100:00100000:1.0:1787826264.354727:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9290 00000100:00100000:1.0:1787826264.354738:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223173:x1874663332504704:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:1.0:1787826264.355035:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223173:x1874663332504704:12345-192.168.203.56@tcp:35:bash.0 Request processed in 297us (542us total) trans 8589957294 rc 0/0 00000100:00100000:1.0:1787826264.355043:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9290 00000100:00100000:0.0:1787826264.395778:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826264.396173:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332504832 00000100:00100000:3.0:1787826264.396183:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332504832 00000100:00100000:3.0:1787826264.396265:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10969 00000100:00100000:3.0:1787826264.396276:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:6166:x1874663332504832:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826264.396291:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826264.396318:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49164 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dc6 expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.396339:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:3.0:1787826264.396356:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f490d8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d95 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.396366:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826264.396375:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f490f4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d9c expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.396385:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7845cb1c00) 00010000:00010000:3.0:1787826264.396391:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826264.396459:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:6166:x1874663332504832:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 184us (691us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.396468:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10969 00000100:00100000:2.0:1787826264.396793:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826264.396869:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332504960 00000100:00100000:0.0:1787826264.396875:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332504960 00000100:00100000:0.0:1787826264.396918:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10970 00000100:00100000:0.0:1787826264.396926:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:6167:x1874663332504960:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826264.396935:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826264.396949:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f48f57 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1db2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d48 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826264.396962:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db2:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826264.396971:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826264.397013:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:6167:x1874663332504960:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 88us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826264.397021:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10970 00000100:00100000:0.0:1787826264.403090:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826264.403209:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332505088 00000100:00100000:3.0:1787826264.403219:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332505088 00000100:00100000:3.0:1787826264.403287:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10971 00000100:00100000:3.0:1787826264.403298:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c83850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:6166:x1874663332505088:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826264.403315:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826264.403339:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4913a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800db8 expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.403358:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:3.0:1787826264.403386:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49102 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800da3 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.403396:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81b00) 00010000:00010000:3.0:1787826264.403403:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f48ebd lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800d25 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826264.403416:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3c:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:3.0:1787826264.403422:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826264.403482:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c83850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:6166:x1874663332505088:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 186us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826264.403490:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10971 00000100:00100000:1.0:1787826264.556928:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346086272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826264.557040:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826264.557251:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346086272 00000100:00100000:2.0:1787826264.557259:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346086272 00000100:00100000:2.0:1787826264.557314:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6187 00000100:00100000:2.0:1787826264.557324:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346086272:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1787826264.557353:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3914711040 avail 3683188736 00000100:00100000:2.0:1787826264.557408:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346086272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (368us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826264.557415:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6187 00000100:00100000:3.0:1787826264.557653:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346086272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826265.068700:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346086400:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826265.068774:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826265.068822:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826265.068829:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346086528:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826265.068860:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826265.068895:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346086400 00000100:00100000:1.0:1787826265.068901:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346086400 00000100:00100000:1.0:1787826265.068967:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6188 00000100:00100000:1.0:1787826265.068986:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346086400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826265.069016:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826265.069059:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346086400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826265.069065:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6188 00000100:00100000:1.0:1787826265.069088:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346086528 00000100:00100000:1.0:1787826265.069091:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346086528 00000100:00100000:1.0:1787826265.069455:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23194 00000100:00100000:1.0:1787826265.069465:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79678fb850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346086528:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826265.069499:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443442688 avail 1297203200 00000100:00100000:1.0:1787826265.069537:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79678fb850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346086528:12345-0@lo:41:osp-pre-1-0.0 Request processed in 75us (675us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826265.069543:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23194 00000100:00100000:1.0:1787826265.069799:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346086528:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826265.069815:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346086400:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1787826265.069857:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826265.074430:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346086784:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826265.074507:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826265.074555:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826265.074563:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346086656:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826265.074631:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826265.074664:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346086784 00000100:00100000:3.0:1787826265.074672:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346086784 00000100:00100000:3.0:1787826265.074718:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6189 00000100:00100000:3.0:1787826265.074727:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346086784:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826265.074748:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:3.0:1787826265.074789:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346086784:12345-0@lo:13:osp-pre-0-1.0 Request processed in 62us (282us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826265.074795:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6189 00000100:00100000:3.0:1787826265.074839:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346086784:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826265.074855:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826265.074862:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346086912:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826265.074882:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826265.074905:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346086912 00000100:00100000:3.0:1787826265.074908:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346086912 00000100:00100000:3.0:1787826265.074926:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6190 00000100:00100000:3.0:1787826265.074932:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346086912:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787826265.074941:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3914711040 avail 3683188736 00000100:00100000:3.0:1787826265.074967:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346086912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 35us (85us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826265.074985:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6190 00000100:00100000:3.0:1787826265.075012:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346086912:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826265.075031:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346086656 00000100:00100000:3.0:1787826265.075035:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346086656 00000100:00100000:3.0:1787826265.075059:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23195 00000100:00100000:3.0:1787826265.075065:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346086656:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787826265.075088:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442832384 avail 1296592896 00000100:00100000:3.0:1787826265.075170:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346086656:12345-0@lo:41:osp-pre-0-1.0 Request processed in 104us (537us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826265.075177:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23195 00000100:00100000:3.0:1787826265.075204:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346086656:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1787826265.261409:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826265.869064:0:164491:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:3.0:1787826265.869650:0:164491:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:1.0:1787826266.289761:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826267.308993:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826267.630766:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826267.630785:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826267.630830:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826267.630837:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826267.630875:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f48ff1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f48ff8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826267.630896:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985312 is committed 00000020:00080000:2.0:1787826267.630910:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7848d58840 refcount 2 committed 0 result 0 batchid 3385 00000020:00080000:2.0:1787826267.630914:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7848d58000 00000020:00080000:2.0:1787826267.630917:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1487 00000020:00080000:2.0:1787826267.630919:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784ac99400 00000020:00080000:2.0:1787826267.630922:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1169 00000001:00080000:2.0:1787826267.630932:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985309 is committed 00000020:00080000:2.0:1787826267.630936:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7848d5e900 refcount 2 committed 0 result 0 batchid 3384 00000020:00080000:2.0:1787826267.630938:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7848d5e6c0 00000020:00080000:2.0:1787826267.630941:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1486 00000020:00080000:2.0:1787826267.630942:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7846231000 00000020:00080000:2.0:1787826267.630945:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1168 00000004:00080000:2.0:1787826267.630967:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 397 committed 00000020:00080000:2.0:1787826267.630977:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784d02da80 refcount 2 committed 0 result 0 batchid 3383 00000020:00080000:2.0:1787826267.630980:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796efc5000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d02d780 00000020:00080000:2.0:1787826267.630983:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1485 00000020:00080000:2.0:1787826267.630985:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433da500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784b769a00 00000020:00080000:2.0:1787826267.630988:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1167 00000020:00080000:2.0:1787826267.630995:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784d02dc00 refcount 2 committed 0 result 0 batchid 3382 00000020:00080000:2.0:1787826267.630997:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433da200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d02d9c0 00000020:00080000:2.0:1787826267.631000:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1484 00000020:00080000:2.0:1787826267.631002:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433da700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7846231800 00000020:00080000:2.0:1787826267.631005:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1166 00000001:00080000:2.0:1787826267.631012:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985303 is committed 00000020:00080000:2.0:1787826267.631022:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3381 batchid updated 00000100:00100000:0.0:1787826267.631069:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826267.631090:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346087040:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826267.631182:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:3.0:1787826267.631306:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3385 committed batchid 3381 00000004:00080000:3.0:1787826267.631406:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78524050c0 with version 6770 00000020:00080000:3.0:1787826267.631423:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3380 cancel update log [0x200000401:0x3:0x0].1482: rc = 0 00000020:00080000:3.0:1787826267.631465:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3380 cancel update log [0x240000400:0x3:0x0].1164: rc = 0 00000004:00080000:3.0:1787826267.631487:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852405300 with version 6771 00000020:00080000:3.0:1787826267.631491:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3381 cancel update log [0x200000401:0x3:0x0].1483: rc = 0 00000020:00080000:3.0:1787826267.631511:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3381 cancel update log [0x240000400:0x3:0x0].1165: rc = 0 00000020:00080000:3.0:1787826267.631519:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3381 00000100:00100000:3.0:1787826267.631584:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346087168:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1787826267.631618:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826267.631661:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346087168 00000100:00100000:3.0:1787826267.631669:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346087168 00000100:00100000:3.0:1787826267.631716:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30767 00000100:00100000:3.0:1787826267.631724:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346087168:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:3.0:1787826267.631747:0:8444:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1222 count 1 00010000:00010000:3.0:1787826267.631767:0:8444:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.631786:0:8444:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.631795:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4c6:0x0].0x0 since=10s v=(0 0) 00010000:00010000:3.0:1787826267.631799:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826267.631803:0:8444:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.631830:0:8444:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.631843:0:8444:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 6 pid: 17826 timeout: 8179 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826267.631853:0:8444:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8444:1874663346087296:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826267.631905:0:8444:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405900 going to sleep for 11 seconds 00000004:00080000:0.0:1787826267.632064:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6770 rpc_version 6770 00000020:00100000:0.0:1787826267.632084:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826267.632091:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b759c00 x1874663346087552/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826267.632135:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346087552:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826267.632165:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826267.632182:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826267.632416:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346087040 00000100:00100000:2.0:1787826267.632422:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346087040 00000100:00100000:2.0:1787826267.632461:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10972 00000100:00100000:2.0:1787826267.632468:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346087040:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826267.632478:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826267.632497:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f48ff8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f48ff1 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826267.632512:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18a:0x0].0x5a734ce9 (ffff9f79460a2a00) 00010000:00010000:2.0:1787826267.632522:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826267.632563:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346087040:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 95us (1380us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826267.632569:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10972 00000100:00100000:2.0:1787826267.632616:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346087040:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787826267.632771:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346087552 00000100:00100000:3.0:1787826267.632775:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346087552 00000100:00100000:3.0:1787826267.632810:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23196 00000100:00100000:3.0:1787826267.632817:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346087552:12345-0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826267.633009:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633020:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633031:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633037:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826267.633042:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633049:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633060:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81b00) 00010000:00010000:1.0:1787826267.633063:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826267.633067:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633073:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f490b5 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633089:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633096:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633101:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633132:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826267.633135:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633144:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633152:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7961d81000) 00010000:00010000:1.0:1787826267.633154:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826267.633157:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633161:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f490bc lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633181:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633188:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633193:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633197:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826267.633200:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633206:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633212:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f7961d81400) 00010000:00010000:1.0:1787826267.633215:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826267.633218:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826267.633234:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f49133 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826267.633256:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826267.633316:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346087552:12345-0@lo:1000:osp_up0-1.0 Request processed in 498us (1148us total) trans 8589957295 rc 0/0 00000100:00100000:3.0:1787826267.633324:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23196 00000100:00100000:0.0:1787826267.633963:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346087552:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826267.633979:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6771 rpc_version 6771 00000020:00100000:0.0:1787826267.633997:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826267.634003:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b75a300 x1874663346087808/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826267.634023:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346087808:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826267.634065:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826267.634241:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826267.634381:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346087808 00000100:00100000:3.0:1787826267.634387:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346087808 00000100:00100000:3.0:1787826267.634434:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23197 00000100:00100000:3.0:1787826267.634443:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346087808:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826267.634583:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346087808:12345-0@lo:1000:osp_up0-1.0 Request processed in 140us (516us total) trans 8589957296 rc 0/0 00000100:00100000:3.0:1787826267.634590:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23197 00000100:00100000:2.0:1787826267.634667:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346087808:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826267.638760:0:8444:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8444:1874663346087296:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826267.638795:0:8444:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826267.640507:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826267.640678:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332505216 00000100:00100000:3.0:1787826267.640687:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332505216 00010000:00010000:3.0:1787826267.640740:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 7 pid: 17826 timeout: 8179 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826267.640779:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21823 00000100:00080000:1.0:1787826267.640781:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826267.640790:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332505216:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826267.640805:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826267.640811:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 7 pid: 17826 timeout: 8179 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.640825:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 7 pid: 17826 timeout: 8179 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.640846:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000f048630/0xe21309aa81f48e1c lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ce6 expref: 7 pid: 17826 timeout: 8179 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.640863:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826267.640867:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c6:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826267.640872:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.640882:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826267.640916:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826267.640980:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332505216:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 193us (491us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826267.640988:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21823 00010000:00010000:0.0:1787826267.642450:0:8444:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826267.642468:0:8444:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826267.642481:0:8444:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826267.642490:0:8444:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826267.642517:0:8444:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826267.642697:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346087168:12345-0@lo:6:osp-syn-1-1.0 Request processed in 10971us (11075us total) trans 4294976141 rc 0/0 00000100:00100000:0.0:1787826267.642706:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30767 00010000:00010000:2.0:1787826267.642770:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826267.642778:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826267.642786:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826267.642790:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826267.642798:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826267.642815:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826267.642823:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4aeb212/0xe21309aa81f49179 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4c6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826267.642841:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826267.642843:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826267.642845:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1787826267.643032:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7842c93100/2, rc 0, transno 4294976141 00000100:00100000:3.0:1787826267.643040:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346087168:0@lo:6:osp-syn-1-1.0 00010000:00010000:0.0:1787826268.332755:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826268.386670:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.386706:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.386734:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.386754:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.386792:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.386811:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.386843:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.386890:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.386918:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332505472 00000100:00100000:0.0:1787826268.386928:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332505472 00000100:00100000:0.0:1787826268.387000:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332505984 00000100:00100000:0.0:1787826268.387004:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332505984 00000100:00100000:0.0:1787826268.387021:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332505600 00000100:00100000:0.0:1787826268.387024:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332505600 00000100:00100000:0.0:1787826268.387040:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332505856 00000100:00100000:0.0:1787826268.387042:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332505856 00000100:00100000:0.0:1787826268.387060:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14211 00000100:00100000:0.0:1787826268.387070:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332505472:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826268.387161:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332505472:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 91us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826268.387170:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14211 00000100:00100000:0.0:1787826268.387191:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14212 00000100:00100000:0.0:1787826268.387198:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332505984:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826268.387239:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332505984:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 42us (504us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826268.387246:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14212 00000100:00100000:0.0:1787826268.387265:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14213 00000100:00100000:0.0:1787826268.387272:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332505600:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826268.387297:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332505600:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (485us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826268.387302:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14213 00000100:00100000:0.0:1787826268.387313:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14214 00000100:00100000:0.0:1787826268.387319:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332505856:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826268.387343:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332505856:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826268.387349:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14214 00000100:00080000:0.0:1787826268.387371:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787826268.387374:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826268.389395:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332505728 00000100:00100000:3.0:1787826268.389410:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332505728 00000100:00100000:3.0:1787826268.389496:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332506240 00000100:00100000:3.0:1787826268.389500:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332506240 00000100:00100000:3.0:1787826268.389521:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332506112 00000100:00100000:3.0:1787826268.389525:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332506112 00000100:00100000:3.0:1787826268.389550:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8749 00000100:00100000:3.0:1787826268.389562:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332505728:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826268.389631:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332505728:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 70us (2924us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826268.389638:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8749 00000100:00100000:3.0:1787826268.389656:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8750 00000100:00100000:3.0:1787826268.389663:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332506240:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826268.389686:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332506240:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (2931us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826268.389693:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8750 00000100:00100000:3.0:1787826268.389705:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8751 00000100:00100000:3.0:1787826268.389712:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332506112:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826268.389735:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332506112:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (2845us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826268.389742:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8751 00000100:00080000:3.0:1787826268.389783:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826268.390100:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332505344 00000100:00100000:2.0:1787826268.390130:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332505344 00000100:00100000:2.0:1787826268.390162:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2204 00000100:00100000:2.0:1787826268.390169:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847250850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332505344:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826268.390197:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847250850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332505344:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (3404us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826268.390204:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2204 00000001:00080000:2.0:1787826268.401397:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976141 is committed 00000001:00100000:2.0:1787826268.401413:0:9524:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:2.0:1787826268.401416:0:9524:0:(genops.c:885:class_export_put()) final put ffff9f79430d6800/398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:3.0:1787826268.406063:0:21470:0:(genops.c:844:class_export_destroy()) destroying export ffff9f79430d6800/398e0287-d3c7-44dd-b1d9-6b85f698e615 for lustre-OST0001 00000020:00000080:3.0:1787826268.406169:0:21470:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9f7842cd9800 for lustre-OST0001 00000100:00100000:1.0:1787826268.974369:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826268.974600:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332506368 00000100:00100000:0.0:1787826268.974610:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332506368 00000100:00100000:0.0:1787826268.974671:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68169 00000100:00100000:0.0:1787826268.974681:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223354:x1874663332506368:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826268.974703:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826268.974725:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f49180 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800dd4 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826268.975559:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826268.975575:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826268.975588:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826268.975633:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826268.975636:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826268.975653:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826268.975680:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f49180 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826268.975696:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826268.975704:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826268.975707:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826268.975711:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826268.975767:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223354:x1874663332506368:12345-192.168.203.56@tcp:101:cat.0 Request processed in 1087us (1406us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826268.975773:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68169 00000100:00100000:0.0:1787826268.992129:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826268.994388:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332506496 00000100:00100000:1.0:1787826268.994401:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332506496 00000100:00100000:1.0:1787826268.994496:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68170 00000100:00100000:1.0:1787826268.994508:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223354:x1874663332506496:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826268.994529:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826268.994551:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4918e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ddb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994620:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994634:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994645:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994659:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994666:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994671:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994737:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f491a3 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994745:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f491a3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994752:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f491a3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826268.994791:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f491a3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994806:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994813:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994818:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994823:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994828:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826268.994832:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994838:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994849:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826268.994853:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826268.994856:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994862:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49195 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994872:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994877:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994883:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994887:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826268.994889:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994894:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994900:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1e00) 00010000:00010000:1.0:1787826268.994902:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826268.994905:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994910:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4919c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994950:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4918e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ddb expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994968:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f491a3 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ddb expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826268.994974:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:1.0:1787826268.994976:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826268.994978:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826268.995057:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223354:x1874663332506496:12345-192.168.203.56@tcp:101:cat.0 Request processed in 550us (2950us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826268.995064:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68170 00000100:00100000:1.0:1787826269.002209:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826269.003443:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332506624 00000100:00100000:3.0:1787826269.003455:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332506624 00000100:00100000:3.0:1787826269.003537:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68171 00000100:00100000:3.0:1787826269.003548:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:223354:x1874663332506624:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826269.003570:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826269.003593:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f491aa lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800de2 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.003654:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f491b1 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.003667:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f491b1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.003681:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f491b1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826269.003771:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3e:0x0]: returning size 4096 00000004:00200000:3.0:1787826269.003777:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826269.004348:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f491aa lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800de2 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.004369:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f491b1 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800de2 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.004379:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:3.0:1787826269.004382:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826269.004385:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68800, rc 0) 00000100:00100000:3.0:1787826269.004460:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223354:x1874663332506624:12345-192.168.203.56@tcp:101:cat.0 Request processed in 913us (2259us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.004469:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68171 00000100:00100000:2.0:1787826269.014772:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826269.014873:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332506752 00000100:00100000:1.0:1787826269.014883:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332506752 00000100:00100000:1.0:1787826269.014953:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68172 00000100:00100000:1.0:1787826269.014965:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223354:x1874663332506752:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826269.014986:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826269.015010:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f491b8 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800de9 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.015070:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.015083:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.015094:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826269.015162:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18d:0x0]: returning size 4096 00000004:00200000:1.0:1787826269.015168:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826269.015212:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f491b8 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.015229:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.015237:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:1.0:1787826269.015240:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826269.015243:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826269.015304:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223354:x1874663332506752:12345-192.168.203.56@tcp:101:cat.0 Request processed in 340us (537us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826269.015315:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68172 00000100:00100000:0.0:1787826269.023501:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826269.023591:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332506880 00000100:00100000:1.0:1787826269.023601:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332506880 00000100:00100000:1.0:1787826269.023661:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68173 00000100:00100000:1.0:1787826269.023672:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223354:x1874663332506880:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826269.023691:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826269.023715:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f491c6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800df0 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.023774:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.023798:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.023813:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826269.023855:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18b:0x0]: returning size 4096 00000004:00200000:1.0:1787826269.023860:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826269.023895:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f491c6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.023912:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.023920:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7845cb1600) 00010000:00010000:1.0:1787826269.023923:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826269.023926:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826269.024003:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223354:x1874663332506880:12345-192.168.203.56@tcp:101:cat.0 Request processed in 328us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826269.024011:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68173 00000100:00100000:3.0:1787826269.033825:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826269.033969:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332507136 00000100:00100000:0.0:1787826269.033981:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332507136 00000100:00100000:0.0:1787826269.034046:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68174 00000100:00100000:0.0:1787826269.034058:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223354:x1874663332507136:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826269.034079:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826269.034100:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f491d4 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800df7 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.034209:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.034224:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.034235:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.034261:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.034268:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.034275:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00020000:0.0:1787826269.034344:0:35093:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000100:00100000:1.0:1787826269.165299:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826269.166082:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346088064:0@lo:400:kworker.0 00000100:00100000:0.0:1787826269.166166:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826269.166182:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346088320:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.166183:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826269.166224:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.166237:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826269.166246:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346088192:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.166273:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.166318:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346088064 00000100:00100000:3.0:1787826269.166325:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346088064 00000100:00100000:3.0:1787826269.166373:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23198 00000100:00100000:3.0:1787826269.166381:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346088064:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.166436:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346088064:12345-0@lo:400:kworker.0 Request processed in 55us (256us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.166443:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23198 00000100:00100000:0.0:1787826269.166487:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346088320 00000100:00100000:0.0:1787826269.166492:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346088320 00000100:00100000:3.0:1787826269.166493:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346088064:0@lo:400:kworker.0 00000100:00080000:3.0:1787826269.166524:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826269.166534:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346088192 00000100:00100000:0.0:1787826269.166539:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346088192 00000100:00100000:3.0:1787826269.166553:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14215 00000100:00100000:3.0:1787826269.166562:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346088320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826269.166574:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14216 00000100:00100000:0.0:1787826269.166584:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346088192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.166607:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346088320:12345-0@lo:400:kworker.0 Request processed in 46us (382us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.166615:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14215 00000100:00100000:0.0:1787826269.166635:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346088192:12345-0@lo:400:kworker.0 Request processed in 52us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.166644:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14216 00000100:00100000:3.0:1787826269.166655:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346088320:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.166685:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346088192:0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.169274:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346087936:0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.169355:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.169879:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346088832:0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.169917:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.169956:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346089344:0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.169994:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.170015:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1787826269.170021:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346088448:0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.170045:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.170064:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346088960:0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.170086:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.170122:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346089472:0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.170143:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.170172:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346088832 00000100:00100000:1.0:1787826269.170180:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346088832 00000100:00100000:2.0:1787826269.170313:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346088448 00000100:00100000:2.0:1787826269.170318:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346088448 00000100:00100000:2.0:1787826269.170349:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346089472 00000100:00100000:2.0:1787826269.170352:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346089472 00000100:00100000:2.0:1787826269.170373:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14217 00000100:00100000:2.0:1787826269.170382:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346088448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826269.170441:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346088448:12345-0@lo:400:kworker.0 Request processed in 59us (395us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.170449:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14217 00000100:00100000:2.0:1787826269.170468:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14218 00000100:00100000:2.0:1787826269.170473:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346089472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826269.170499:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346089472:12345-0@lo:400:kworker.0 Request processed in 27us (355us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.170505:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14218 00000100:00100000:2.0:1787826269.170526:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346089344 00000100:00100000:2.0:1787826269.170529:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346089344 00000100:00100000:2.0:1787826269.170554:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346088960 00000100:00100000:2.0:1787826269.170556:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346088960 00000100:00100000:2.0:1787826269.170574:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8752 00000100:00100000:2.0:1787826269.170579:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346089344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826269.170608:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346089344:12345-0@lo:400:kworker.0 Request processed in 29us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.170614:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8752 00000100:00100000:2.0:1787826269.170625:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8753 00000100:00100000:2.0:1787826269.170631:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346088960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826269.170656:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346088960:12345-0@lo:400:kworker.0 Request processed in 25us (569us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.170662:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8753 00000100:00100000:2.0:1787826269.170701:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346088448:0@lo:400:kworker.0 00000100:00100000:2.0:1787826269.170712:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976141 gen 2 00000004:00080000:2.0:1787826269.170717:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7842c93100, transno 4294976141 00000100:00100000:2.0:1787826269.170726:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346088960:0@lo:400:kworker.0 00000100:00100000:2.0:1787826269.170735:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346089472:0@lo:400:kworker.0 00000100:00100000:2.0:1787826269.170743:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346089344:0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.170883:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14219 00000100:00100000:1.0:1787826269.170892:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346088832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.170928:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346088832:12345-0@lo:400:kworker.0 Request processed in 39us (1011us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826269.170934:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14219 00000100:00100000:2.0:1787826269.171086:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346088832:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171303:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346088704:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171337:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.171370:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346089216:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171393:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.171423:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787826269.171430:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346088576:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171453:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.171471:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346089088:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171493:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.171520:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346089216 00000100:00100000:3.0:1787826269.171525:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346089216 00000100:00100000:3.0:1787826269.171554:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346088576 00000100:00100000:3.0:1787826269.171557:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346088576 00000100:00100000:3.0:1787826269.171579:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8754 00000100:00100000:3.0:1787826269.171586:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346089216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171622:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346089216:12345-0@lo:400:kworker.0 Request processed in 37us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.171628:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8754 00000100:00100000:3.0:1787826269.171643:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8755 00000100:00100000:3.0:1787826269.171650:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346088576:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171675:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346088576:12345-0@lo:400:kworker.0 Request processed in 27us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.171681:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8755 00000100:00100000:3.0:1787826269.171723:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346089088 00000100:00100000:3.0:1787826269.171727:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346089088 00000100:00100000:3.0:1787826269.171763:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23199 00000100:00100000:3.0:1787826269.171778:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346089088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171811:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346089088:12345-0@lo:400:kworker.0 Request processed in 34us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.171817:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23199 00000100:00080000:3.0:1787826269.171844:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826269.171849:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826269.171852:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826269.171877:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346088576:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.171885:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985303 gen 2 00000004:00080000:3.0:1787826269.171890:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985302 committed transno 4294985303 00000004:00080000:3.0:1787826269.171906:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985303 committed transno 4294985303 00000020:00080000:3.0:1787826269.171912:0:3655:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f784548a3c0 refcount 2 committed 0 result 0 batchid 4294979074 00000020:00080000:3.0:1787826269.171916:0:3655:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784548a300 00000020:00080000:3.0:1787826269.171919:0:3655:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].930 00000020:00080000:3.0:1787826269.171921:0:3655:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784dac4600 00000020:00080000:3.0:1787826269.171924:0:3655:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].927 00000100:00100000:3.0:1787826269.171990:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346089088:0@lo:400:kworker.0 00000100:00100000:3.0:1787826269.172005:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346089216:0@lo:400:kworker.0 00000020:00080000:3.0:1787826269.172027:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979074 committed batchid 4294979073 00000020:00080000:3.0:1787826269.172064:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979073 00000100:00100000:1.0:1787826269.173409:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346088704 00000100:00100000:1.0:1787826269.173420:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346088704 00000100:00100000:1.0:1787826269.173496:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14220 00000100:00100000:1.0:1787826269.173507:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346088704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826269.173586:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346088704:12345-0@lo:400:kworker.0 Request processed in 81us (2248us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826269.173593:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14220 00000100:00080000:3.0:1787826269.174048:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826269.174086:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346088704:0@lo:400:kworker.0 00000100:00100000:0.0:1787826269.174280:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346087936 00000100:00100000:0.0:1787826269.174287:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346087936 00000100:00100000:0.0:1787826269.174869:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2205 00000100:00100000:0.0:1787826269.174881:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847251850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346087936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826269.174965:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847251850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346087936:12345-0@lo:400:kworker.0 Request processed in 87us (5609us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.174980:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2205 00000100:00100000:3.0:1787826269.175066:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346087936:0@lo:400:kworker.0 00010000:00010000:1.0:1787826269.357565:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826269.411644:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826269.411698:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826269.411806:0:20174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332507264 00000100:00100000:1.0:1787826269.411816:0:20174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332507264 00002000:00100000:1.0:1787826269.411866:0:20174:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x4c9:0x0] (0->5) req@ffff9f784b19df80 x1874663332507264/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:625/0 lens 496/0 e 0 to 0 dl 1787826280 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826269.411876:0:20174:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b19df80 with x1874663332507264 ext(0->5) 00002000:00010000:1.0:1787826269.411882:0:20174:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784b19df80 00000100:00100000:1.0:1787826269.411914:0:20174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27627 00000100:00100000:1.0:1787826269.411925:0:20174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332507264:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826269.411934:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332507392 00000100:00100000:2.0:1787826269.411949:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332507392 00000100:00100000:2.0:1787826269.411991:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30768 00000100:00100000:2.0:1787826269.412000:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332507392:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:2.0:1787826269.412095:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332507392:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 95us (458us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.412129:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30768 00000100:00100000:2.0:1787826269.414411:0:20174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332507264:12345-192.168.203.56@tcp:4:bash.0 Request processed in 2478us (2703us total) trans 4294973833 rc 0/0 00000100:00100000:2.0:1787826269.414423:0:20174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27627 00002000:00100000:2.0:1787826269.414435:0:20174:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x4c9:0x0] (0->5) req@ffff9f784b19df80 x1874663332507264/t4294973833(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:625/0 lens 496/448 e 0 to 0 dl 1787826280 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826269.414452:0:20174:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b19df80 with x1874663332507264 ext(0->5) 00002000:00010000:2.0:1787826269.414466:0:20174:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784b19df80 00000001:00080000:2.0:1787826269.416839:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973833 is committed 00000001:00080000:2.0:1787826269.416855:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973832 is committed 00000100:00100000:0.0:1787826269.425396:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826269.425444:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826269.425627:0:8450:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332507520 00000100:00100000:1.0:1787826269.425637:0:8450:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332507520 00002000:00100000:1.0:1787826269.425707:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x51c:0x0] (0->5) req@ffff9f784b19f800 x1874663332507520/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:625/0 lens 496/0 e 0 to 0 dl 1787826280 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826269.425718:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b19f800 with x1874663332507520 ext(0->5) 00002000:00010000:1.0:1787826269.425724:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784b19f800 00000100:00100000:1.0:1787826269.425762:0:8450:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27628 00000100:00100000:1.0:1787826269.425772:0:8450:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332507520:12345-192.168.203.56@tcp:4:bash.0 00000100:00080000:3.0:1787826269.426263:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826269.426693:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332507648 00000100:00100000:2.0:1787826269.426699:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332507648 00000100:00100000:2.0:1787826269.426749:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30769 00000100:00100000:2.0:1787826269.426759:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332507648:12345-192.168.203.56@tcp:16:kworker.0 00000001:00080000:0.0:1787826269.429619:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976142 is committed 00000100:00100000:2.0:1787826269.429977:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332507648:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 3213us (4526us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.429988:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30769 00000100:00100000:2.0:1787826269.432474:0:8450:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332507520:12345-192.168.203.56@tcp:4:bash.0 Request processed in 6685us (7070us total) trans 4294976142 rc 0/0 00000100:00100000:2.0:1787826269.432488:0:8450:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27628 00002000:00100000:2.0:1787826269.432500:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x51c:0x0] (0->5) req@ffff9f784b19f800 x1874663332507520/t4294976142(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:625/0 lens 496/448 e 0 to 0 dl 1787826280 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826269.432511:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b19f800 with x1874663332507520 ext(0->5) 00002000:00010000:2.0:1787826269.432517:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784b19f800 00000100:00100000:0.0:1787826269.518743:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826269.520041:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332507776 00000100:00100000:3.0:1787826269.520052:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332507776 00000100:00100000:3.0:1787826269.520136:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68175 00000100:00100000:3.0:1787826269.520146:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223356:x1874663332507776:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826269.520167:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826269.520188:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f491e9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800dfe expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.520253:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.520267:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.520278:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826269.520327:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826269.520330:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826269.520351:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.520374:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f491e9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.520391:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.520399:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826269.520402:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826269.520405:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826269.520751:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223356:x1874663332507776:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 604us (2025us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.520759:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68175 00000100:00100000:1.0:1787826269.525955:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826269.526213:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332507904 00000100:00100000:0.0:1787826269.526223:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332507904 00000100:00100000:0.0:1787826269.526330:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68176 00000100:00100000:0.0:1787826269.526340:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223356:x1874663332507904:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826269.526361:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826269.526381:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f491f7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e05 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526441:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526454:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526474:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526488:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526494:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526500:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526581:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526589:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526595:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826269.526616:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526629:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526635:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526640:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526646:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526651:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826269.526655:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526661:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526672:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826269.526675:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826269.526680:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526685:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f491fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526698:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526704:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526710:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526715:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826269.526718:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526723:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526739:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a800) 00010000:00010000:0.0:1787826269.526741:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826269.526744:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526749:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49205 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.526779:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f491f7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.527573:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.527584:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:0.0:1787826269.527586:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826269.527588:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826269.527658:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223356:x1874663332507904:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1319us (1723us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.527667:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68176 00000100:00100000:0.0:1787826269.536968:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826269.537047:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332508032 00000100:00100000:3.0:1787826269.537058:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332508032 00000100:00100000:3.0:1787826269.537154:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68177 00000100:00100000:3.0:1787826269.537166:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:223356:x1874663332508032:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826269.537188:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826269.537209:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49213 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e0c expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.537271:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4921a lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.537285:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4921a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.537296:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4921a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826269.537383:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3e:0x0]: returning size 4096 00000004:00200000:3.0:1787826269.537387:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826269.537426:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49213 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e0c expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.537443:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4921a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e0c expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.537451:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:3.0:1787826269.537455:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826269.537470:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826269.537526:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223356:x1874663332508032:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 362us (567us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.537533:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68177 00000100:00100000:1.0:1787826269.547016:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826269.547137:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332508160 00000100:00100000:2.0:1787826269.547148:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332508160 00000100:00100000:2.0:1787826269.547207:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68178 00000100:00100000:2.0:1787826269.547218:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223356:x1874663332508160:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826269.547237:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826269.547258:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49221 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e13 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547322:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547335:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547345:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547357:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547364:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547370:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547451:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49236 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547469:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49236 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547477:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49236 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826269.547518:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db3:0x0]: returning size 6 00000004:00200000:2.0:1787826269.547522:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826269.547540:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49236 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547554:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49236 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547563:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547570:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547576:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547582:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547587:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826269.547592:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547598:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547611:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7845cb1600) 00010000:00010000:2.0:1787826269.547614:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826269.547619:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547625:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49228 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547641:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547647:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547652:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547657:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826269.547659:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547664:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547672:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x4ec94fd6 (ffff9f7845cb1100) 00010000:00010000:2.0:1787826269.547674:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826269.547677:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547683:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4922f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547709:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49221 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e13 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547723:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49236 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e13 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.547739:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:2.0:1787826269.547741:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826269.547745:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826269.547924:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223356:x1874663332508160:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 707us (919us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.547932:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68178 00000100:00100000:1.0:1787826269.555916:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826269.558875:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332508288 00000100:00100000:2.0:1787826269.558890:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332508288 00000100:00100000:2.0:1787826269.559041:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68179 00000100:00100000:2.0:1787826269.559053:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223356:x1874663332508288:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826269.559074:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826269.559096:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4923d lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e1a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559180:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559195:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559218:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559231:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559240:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559247:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559326:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49252 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559335:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49252 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559341:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49252 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826269.559381:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3f:0x0]: returning size 6 00000004:00200000:2.0:1787826269.559386:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826269.559404:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49252 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559416:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49252 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559426:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559432:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559439:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559446:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559452:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826269.559468:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559476:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559488:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:2.0:1787826269.559491:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826269.559496:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559502:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49244 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559516:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559522:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559528:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559533:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826269.559536:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559541:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559548:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f794b57ab00) 00010000:00010000:2.0:1787826269.559551:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826269.559553:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559559:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4924b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559585:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4923d lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e1a expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559599:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49252 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e1a expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.559605:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:2.0:1787826269.559607:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826269.559610:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826269.559665:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223356:x1874663332508288:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 614us (3760us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.559672:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68179 00000100:00100000:0.0:1787826269.566246:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826269.567680:0:6541:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332508672 00000100:00100000:1.0:1787826269.567695:0:6541:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332508672 00000100:00100000:1.0:1787826269.567792:0:6541:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1222 00000100:00100000:1.0:1787826269.567804:0:6541:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223356:x1874663332508672:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:1.0:1787826269.567856:0:6541:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826269.567884:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49252 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e1a expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.567906:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:1.0:1787826269.567919:0:6541:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826269.567991:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346089600:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826269.568054:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.568089:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f34a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826269.568251:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346089600 00000100:00100000:3.0:1787826269.568256:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346089600 00000100:00100000:3.0:1787826269.568294:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23200 00000100:00100000:3.0:1787826269.568301:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346089600:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.568366:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346089600:12345-0@lo:1000:mdt_io00_001.0 Request processed in 65us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.568386:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23200 00000100:00100000:1.0:1787826269.569644:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346089600:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826269.569721:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346089728:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826269.569843:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.569866:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826269.570088:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346089728 00000100:00100000:3.0:1787826269.570095:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346089728 00000100:00100000:3.0:1787826269.570182:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23201 00000100:00100000:3.0:1787826269.570192:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346089728:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.570306:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346089728:12345-0@lo:1000:mdt_io00_001.0 Request processed in 116us (463us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.570314:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23201 00000100:00100000:0.0:1787826269.570436:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346089728:0@lo:1000:mdt_io00_001.0 00000004:00080000:0.0:1787826269.570475:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00000100:00100000:0.0:1787826269.570524:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346089856:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.570555:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826269.570577:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69c180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826269.571360:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346089856 00000100:00100000:3.0:1787826269.571368:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346089856 00000100:00100000:3.0:1787826269.571416:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23202 00000100:00100000:3.0:1787826269.571424:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346089856:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.571528:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346089856:12345-0@lo:1000:mdt_io00_001.0 Request processed in 104us (972us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.571535:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23202 00000100:00100000:2.0:1787826269.571664:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346089856:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826269.571710:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00000100:00100000:2.0:1787826269.571753:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346089984:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826269.571785:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826269.571810:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f2ad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.571912:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346089984 00000100:00100000:0.0:1787826269.571917:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346089984 00000100:00100000:0.0:1787826269.571957:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23203 00000100:00100000:0.0:1787826269.571965:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346089984:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.572022:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346089984:12345-0@lo:1000:mdt_io00_001.0 Request processed in 58us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.572028:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23203 00000100:00100000:2.0:1787826269.572337:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346089984:0@lo:1000:mdt_io00_001.0 00000004:00020000:2.0:1787826269.572372:0:6541:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00000100:00100000:1.0:1787826269.589668:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826269.589801:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332508416 00000100:00100000:3.0:1787826269.589812:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332508416 00000100:00100000:3.0:1787826269.589882:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10973 00000100:00100000:3.0:1787826269.589893:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:2394:x1874663332508416:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:3.0:1787826269.589907:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826269.589936:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49236 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e13 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.589968:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:3.0:1787826269.589981:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826269.590047:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2394:x1874663332508416:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 155us (390us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.590055:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10973 00010000:00010000:2.0:1787826269.598281:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.598308:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.598318:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.598365:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.598381:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 8 pid: 6528 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826269.598398:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346090112:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826269.598483:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a300 going to sleep for 11 seconds 00000004:00020000:1.0:1787826269.598999:0:35093:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4454 00000100:00100000:3.0:1787826269.609994:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826269.610088:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332508800 00000100:00100000:0.0:1787826269.610098:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332508800 00010000:00010000:0.0:1787826269.610174:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 9 pid: 6528 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826269.610212:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21824 00000100:00100000:0.0:1787826269.610237:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332508800:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826269.610254:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826269.610261:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 9 pid: 6528 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.610274:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 9 pid: 6528 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.610302:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:0.0:1787826269.610309:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.610318:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.610326:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.610343:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826269.610395:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332508800:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 174us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.610402:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21824 00010000:00010000:2.0:1787826269.611604:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f491bf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800de9 expref: 8 pid: 6528 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826269.611625:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346090112:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826269.611648:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.611657:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.611663:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.611686:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.611697:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 2/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.611704:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.611722:0:6541:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.611747:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623720:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623752:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623764:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826269.623778:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3f:0x0]: returning size 6 00010000:00010000:0.0:1787826269.623882:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623890:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623901:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623908:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623914:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826269.623919:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623926:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623951:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:0.0:1787826269.623954:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826269.623959:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623966:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f491db lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623980:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623985:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623991:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.623996:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826269.623999:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624005:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624012:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f794b57ab00) 00010000:00010000:0.0:1787826269.624015:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826269.624018:0:35093:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624026:0:35093:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624033:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624061:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624068:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f491e2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624147:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f491d4 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624161:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.624168:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:0.0:1787826269.624170:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826269.624173:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826269.624247:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223354:x1874663332507136:12345-192.168.203.56@tcp:101:cat.0 Request processed in 590191us (590431us total) trans 4294985313 rc 0/0 00000100:00100000:0.0:1787826269.624257:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68174 00010000:00010000:2.0:1787826269.624369:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.624377:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.624419:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.624427:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.624436:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826269.624467:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346090240:0@lo:101:mdt_io00_001.0 00000100:00100000:2.0:1787826269.624515:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826269.624589:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.625860:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346090240 00000100:00100000:0.0:1787826269.625872:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346090240 00000100:00100000:0.0:1787826269.625934:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23204 00000100:00100000:0.0:1787826269.625956:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346090240:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826269.625975:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826269.625996:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49275 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.626024:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.626034:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 4/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49275 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.626061:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.626074:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 8 pid: 35080 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826269.626086:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346090368:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826269.626163:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69c180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826269.641787:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826269.641920:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346090368:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826269.641970:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49275 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.641984:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7845cb1600) 00010000:00010000:2.0:1787826269.641988:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826269.641993:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49275 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.642004:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:2.0:1787826269.642063:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346090240:12345-0@lo:101:mdt_io00_001.0 Request processed in 16108us (17546us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.642071:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23204 00000100:00100000:3.0:1787826269.643418:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332508928 00000100:00100000:3.0:1787826269.643431:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332508928 00010000:00010000:3.0:1787826269.643477:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 9 pid: 35080 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826269.643510:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21825 00000100:00100000:3.0:1787826269.643520:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332508928:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826269.643535:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826269.643542:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 9 pid: 35080 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.643554:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 9 pid: 35080 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.643576:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f491cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df0 expref: 9 pid: 35080 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.643594:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7845cb1600) 00010000:00010000:3.0:1787826269.643600:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49275 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.643611:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49275 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.643618:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49275 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.643650:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49275 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826269.643665:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346090496:0@lo:105: 00000100:00100000:3.0:1787826269.643721:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.643914:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346090240:0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826269.643934:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.643954:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826269.644048:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826269.644354:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346090496 00000100:00100000:1.0:1787826269.644360:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346090496 00000100:00100000:1.0:1787826269.644411:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7796 00000100:00100000:1.0:1787826269.644419:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346090496:12345-0@lo:105: 00010000:00010000:1.0:1787826269.644429:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644438:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644446:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644467:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644474:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644482:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644500:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ee600) 00000100:00100000:1.0:1787826269.644542:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346090496:12345-0@lo:105: Request processed in 123us (820us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826269.644549:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7796 00010000:00010000:1.0:1787826269.644571:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644580:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826269.644603:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00010000:00010000:1.0:1787826269.644638:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644645:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.644653:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826269.644671:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346090624:0@lo:101:mdt_io00_001.0 00000100:00100000:1.0:1787826269.644712:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.644738:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f34a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826269.645074:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346090624 00000100:00100000:2.0:1787826269.645080:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346090624 00000100:00100000:2.0:1787826269.645132:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23205 00000100:00100000:2.0:1787826269.645140:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346090624:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:2.0:1787826269.645158:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826269.645173:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49283 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.645192:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 2/0,1 mode: TXN/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.645200:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 4/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49283 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.645226:0:8152:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00080000:00080000:2.0:1787826269.645258:0:8152:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00010000:00010000:2.0:1787826269.645293:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 3/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49283 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.645307:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x4ec94fd6 (ffff9f7845cb1100) 00010000:00010000:2.0:1787826269.645309:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826269.645312:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 3/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49283 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.645319:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826269.645354:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346090624:12345-0@lo:101:mdt_io00_001.0 Request processed in 215us (641us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.645360:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23205 00000100:00100000:2.0:1787826269.645414:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826269.645441:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332509056 00000100:00100000:2.0:1787826269.645445:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332509056 00000100:00100000:2.0:1787826269.645476:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30770 00000100:00100000:2.0:1787826269.645486:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332509056:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826269.645494:0:8444:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826269.645504:0:8444:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f49291 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e21 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826269.645539:0:8444:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8444:1874663346090752:192.168.203.56@tcp:106: 00000100:00100000:2.0:1787826269.645597:0:8444:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548af00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826269.647897:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346090496:0@lo:105: 00010000:00010000:3.0:1787826269.647929:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826269.648010:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332508928:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 4488us (6231us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.648021:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21825 00000100:00100000:1.0:1787826269.648211:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346090624:0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826269.648232:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.648244:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826269.652991:0:8444:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4c9:0x0] updating lvb size: 0 -> 6 00002000:00010000:3.0:1787826269.652999:0:8444:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4c9:0x0] updating lvb mtime: 0 -> 1787826263 00002000:00010000:3.0:1787826269.653003:0:8444:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4c9:0x0] updating lvb atime: 0 -> 1787826263 00002000:00010000:3.0:1787826269.653005:0:8444:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4c9:0x0] updating lvb ctime: 0 -> 1787826263 00002000:00010000:3.0:1787826269.653019:0:8444:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4c9:0x0] updating lvb blocks from disk: 0 -> 8 00000100:00100000:3.0:1787826269.653030:0:8444:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8444:1874663346090752:192.168.203.56@tcp:106: 00010000:00010000:3.0:1787826269.653070:0:8444:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f49291 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e21 expref: 6 pid: 8444 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.653130:0:8444:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f49291 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e21 expref: 6 pid: 8444 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.653146:0:8444:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:3.0:1787826269.653213:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332509056:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7723us (7793us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826269.653223:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30770 00000020:00080000:3.0:1787826269.656534:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979074 batchid updated 00000001:00080000:3.0:1787826269.656568:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957296 is committed 00000001:00080000:3.0:1787826269.656593:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957294 is committed 00010000:00010000:2.0:1787826269.656659:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.656673:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.656685:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.656692:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826269.656698:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.656706:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.656733:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x4ec94fd6 (ffff9f7845cb1100) 00010000:00010000:2.0:1787826269.656736:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826269.656744:0:3639:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49283 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.656756:0:3639:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49283 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.656765:0:3639:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49283 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.656809:0:3639:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49283 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826269.656825:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f540380 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3639:1874663346090880:0@lo:105: 00000100:00100000:2.0:1787826269.656892:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826269.656949:0:3639:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a9c0 going to sleep for 11 seconds 00000020:00080000:1.0:1787826269.657962:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979074 00000004:00080000:1.0:1787826269.658029:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7852f34a80 with version 1810 00000020:00080000:1.0:1787826269.658050:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979074 cancel update log [0x240000404:0x2:0x0].930: rc = 0 00000020:00080000:1.0:1787826269.658174:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979074 cancel update log [0x200000bd0:0x2:0x0].927: rc = 0 00000020:00080000:1.0:1787826269.658182:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979074 00000100:00100000:3.0:1787826269.658322:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826269.658340:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346090880 00000100:00100000:3.0:1787826269.658348:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346090880 00000100:00100000:3.0:1787826269.658469:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7797 00000100:00100000:3.0:1787826269.658480:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b037850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:3639:x1874663346090880:12345-0@lo:105: 00010000:00010000:3.0:1787826269.658495:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.658506:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.658515:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.658524:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 5/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.658532:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.658541:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.658561:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecfa00) 00000100:00100000:3.0:1787826269.658620:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b037850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:3639:x1874663346090880:12345-0@lo:105: Request processed in 142us (1728us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.658628:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7797 00000004:00080000:0.0:1787826269.659054:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1810 rpc_version 1810 00000020:00100000:0.0:1787826269.659080:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826269.659089:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b19f480 x1874663346091136/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826269.659134:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346091136:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826269.659177:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826269.659202:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cb40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826269.659319:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346091136 00000100:00100000:3.0:1787826269.659325:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346091136 00000100:00100000:3.0:1787826269.659366:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23206 00000100:00100000:3.0:1787826269.659373:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:30909:x1874663346091136:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826269.659499:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346091136:12345-0@lo:1000:osp_up1-0.0 Request processed in 125us (321us total) trans 4294985314 rc 0/0 00000100:00100000:3.0:1787826269.659506:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23206 00000100:00100000:0.0:1787826269.659995:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346091136:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826269.660630:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f540380 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3639:1874663346090880:0@lo:105: 00010000:00010000:2.0:1787826269.660654:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.660664:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4915d lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.660698:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.660707:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826269.660736:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00000100:00100000:2.0:1787826269.660783:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f540380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091264:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826269.660828:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826269.660856:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.661158:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346091264 00000100:00100000:0.0:1787826269.661165:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346091264 00000100:00100000:0.0:1787826269.661216:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23207 00000100:00100000:0.0:1787826269.661225:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091264:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.661306:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091264:12345-0@lo:1000:mdt_io00_001.0 Request processed in 82us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.661313:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23207 00000100:00100000:3.0:1787826269.661506:0:6541:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957296 gen 2 00000004:00080000:3.0:1787826269.661512:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957295 committed transno 8589957296 00000004:00080000:3.0:1787826269.661527:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957296 committed transno 8589957296 00000100:00100000:3.0:1787826269.661541:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f540380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091264:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.661590:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b306a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091392:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.661619:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.661636:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.661749:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346091392 00000100:00100000:0.0:1787826269.661753:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346091392 00000100:00100000:0.0:1787826269.661786:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23208 00000100:00100000:0.0:1787826269.661793:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091392:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.661863:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091392:12345-0@lo:1000:mdt_io00_001.0 Request processed in 69us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.661869:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23208 00000100:00100000:3.0:1787826269.662045:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b306a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091392:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.662087:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b306a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091520:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.662180:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.663343:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826269.666185:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826269.666359:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332509184 00000100:00100000:3.0:1787826269.666369:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332509184 00000100:00100000:3.0:1787826269.666420:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30771 00000100:00100000:3.0:1787826269.666429:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:223354:x1874663332509184:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826269.666447:0:8443:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826269.666476:0:8443:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e28 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.666507:0:8443:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.666515:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4c9:0x0].0x0 since=5s v=(0 0) 00010000:00010000:3.0:1787826269.666519:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826269.666525:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 6 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.666558:0:8443:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.666571:0:8443:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 5 pid: 8445 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826269.666585:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346091648:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826269.666672:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78556679c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826269.671010:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346091520 00000100:00100000:1.0:1787826269.671027:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346091520 00000100:00100000:0.0:1787826269.671174:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23209 00000100:00100000:0.0:1787826269.671185:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091520:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.671358:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091520:12345-0@lo:1000:mdt_io00_001.0 Request processed in 174us (9177us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826269.671366:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23209 00000100:00100000:0.0:1787826269.671415:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b306a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091520:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.671524:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091776:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.671562:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826269.671590:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346091776 00000100:00100000:0.0:1787826269.671594:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346091776 00000100:00100000:0.0:1787826269.671617:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23210 00000100:00100000:0.0:1787826269.671623:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091776:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.671672:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091776:12345-0@lo:1000:mdt_io00_001.0 Request processed in 48us (110us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.671678:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23210 00000100:00100000:0.0:1787826269.671696:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69c600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.671708:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091776:0@lo:1000:mdt_io00_001.0 00010000:00010000:0.0:1787826269.671819:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.671829:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.671838:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826269.671850:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091904:0@lo:101:mdt_io00_001.0 00000100:00100000:0.0:1787826269.671874:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826269.671892:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346091904 00000100:00100000:0.0:1787826269.671896:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346091904 00000100:00100000:0.0:1787826269.671918:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23211 00000100:00100000:0.0:1787826269.671925:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346091904:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826269.671933:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826269.671961:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f492a6 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4929f expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.671984:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f492a6 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4929f expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.671997:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f492a6 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4929f expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.672005:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826269.672007:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787826269.672011:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826269.672036:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346091904:12345-0@lo:101:mdt_io00_001.0 Request processed in 112us (161us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.672041:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23211 00000100:00100000:0.0:1787826269.672059:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.672071:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346091904:0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826269.672082:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f492a6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.672092:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f492a6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826269.672125:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000403:0x1db3:0x0] 00000100:00100000:0.0:1787826269.672154:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092032:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.672176:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826269.672195:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346092032 00000100:00100000:0.0:1787826269.672199:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346092032 00000100:00100000:0.0:1787826269.672220:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23212 00000100:00100000:0.0:1787826269.672225:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346092032:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.672275:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346092032:12345-0@lo:1000:mdt_io00_001.0 Request processed in 49us (97us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.672281:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23212 00000100:00100000:0.0:1787826269.672298:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69c600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.672309:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092032:0@lo:1000:mdt_io00_001.0 00000004:00080000:0.0:1787826269.672327:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00010000:00010000:0.0:1787826269.672351:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.672364:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.672373:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.672393:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.672404:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 7 pid: 35093 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826269.672415:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346092160:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826269.672468:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cb40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826269.674088:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346091648:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826269.674151:0:8443:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 6 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.674172:0:8443:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826269.674239:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223354:x1874663332509184:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7810us (8063us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.674247:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30771 00000100:00100000:3.0:1787826269.676969:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346092288:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826269.677047:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.677144:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346092288 00000100:00100000:1.0:1787826269.677153:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346092288 00000100:00100000:1.0:1787826269.677222:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6191 00000100:00100000:1.0:1787826269.677233:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346092288:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826269.677257:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3914715136 avail 3683164160 00000100:00100000:1.0:1787826269.680136:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346092288:12345-0@lo:13:osp-pre-1-1.0 Request processed in 2873us (3057us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826269.680152:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6191 00000100:00100000:2.0:1787826269.680704:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826269.680821:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346092160:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826269.680849:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826269.680988:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826269.681064:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346092288:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826269.681341:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332509312 00000100:00100000:1.0:1787826269.681349:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332509312 00000100:00100000:0.0:1787826269.681659:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332509440 00000100:00100000:0.0:1787826269.681664:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332509440 00010000:00010000:0.0:1787826269.681695:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 8 pid: 35093 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826269.681800:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21826 00000100:00100000:0.0:1787826269.681808:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c80450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332509440:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826269.681821:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826269.681826:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 8 pid: 35093 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.681834:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 8 pid: 35093 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.681850:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49267 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800df7 expref: 8 pid: 35093 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.681861:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:0.0:1787826269.681864:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.681870:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.681875:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.681895:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826269.681948:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c80450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332509440:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 140us (960us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.681956:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21826 00010000:00010000:3.0:1787826269.682002:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.682010:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.682071:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.682080:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x2 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.682090:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x2 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826269.682128:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092416:0@lo:101:mdt_io00_001.0 00000100:00100000:3.0:1787826269.682186:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.682217:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78556679c0 going to sleep for 11 seconds 00010000:00010000:0.0:1787826269.682993:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 6 pid: 8445 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826269.683022:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346092416 00000100:00100000:0.0:1787826269.683029:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21827 00000100:00100000:1.0:1787826269.683029:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346092416 00000100:00100000:0.0:1787826269.683038:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332509312:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826269.683049:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826269.683062:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23213 00000100:00100000:1.0:1787826269.683068:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346092416:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826269.683071:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 6 pid: 8445 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.683083:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 6 pid: 8445 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.683084:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826269.683101:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f49148 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dbf expref: 6 pid: 8445 timeout: 8181 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.683101:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f492bb lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f492b4 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.683132:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826269.683136:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826269.683138:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f492bb lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f492b4 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.683144:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.683150:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f492bb lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f492b4 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.683153:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.683157:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7845cb1400) 00010000:00010000:1.0:1787826269.683160:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826269.683163:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:1.0:1787826269.683213:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346092416:12345-0@lo:101:mdt_io00_001.0 Request processed in 145us (1027us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826269.683219:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23213 00010000:00010000:0.0:1787826269.683235:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826269.683265:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092416:0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826269.683280:0:6541:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 4/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.683290:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.683305:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826269.683373:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6512:1874663346092544:192.168.203.56@tcp:105: 00000100:00100000:1.0:1787826269.683399:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092672:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826269.683425:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826269.683431:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a69cb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.683444:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346092672 00000100:00100000:0.0:1787826269.683448:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346092672 00000100:00100000:1.0:1787826269.683450:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.683479:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23214 00000100:00100000:0.0:1787826269.683486:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346092672:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.684860:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346092672:12345-0@lo:1000:mdt_io00_001.0 Request processed in 1366us (1426us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826269.684867:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092672:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.684873:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23214 00000100:00100000:1.0:1787826269.684950:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092800:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826269.684997:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.685023:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.685045:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346092800 00000100:00100000:0.0:1787826269.685051:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346092800 00000100:00100000:0.0:1787826269.685091:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23215 00000100:00100000:0.0:1787826269.685099:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346092800:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.685944:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346092800:12345-0@lo:1000:mdt_io00_001.0 Request processed in 832us (933us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.685954:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23215 00000100:00100000:3.0:1787826269.686190:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092800:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826269.686237:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00000100:00100000:3.0:1787826269.686305:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092928:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.686345:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826269.686385:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346092928 00000100:00100000:0.0:1787826269.686392:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346092928 00000100:00100000:0.0:1787826269.686440:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23216 00000100:00100000:0.0:1787826269.686448:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346092928:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.686509:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346092928:12345-0@lo:1000:mdt_io00_001.0 Request processed in 62us (164us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.686516:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23216 00000100:00100000:3.0:1787826269.686671:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826269.686689:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346092928:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.686798:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346093056:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.686826:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.686847:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826269.686973:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346093056 00000100:00100000:0.0:1787826269.686979:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346093056 00000100:00100000:0.0:1787826269.687012:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23217 00000100:00100000:0.0:1787826269.687019:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346093056:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.687083:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346093056:12345-0@lo:1000:mdt_io00_001.0 Request processed in 64us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.687090:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23217 00000100:00100000:3.0:1787826269.687155:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346093056:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.687290:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346093184:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826269.687315:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.687380:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826269.690323:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826269.690359:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332509568 00000100:00100000:3.0:1787826269.690368:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332509568 00000100:00100000:3.0:1787826269.690447:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68180 00000100:00100000:3.0:1787826269.690469:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223354:x1874663332509568:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826269.690498:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826269.690519:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f492c2 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e2f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.690577:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f492c2 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e2f expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.690592:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f492c2 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e2f expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.690609:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:3.0:1787826269.690612:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787826269.690616:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826269.690666:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223354:x1874663332509568:12345-192.168.203.56@tcp:101:cat.0 Request processed in 209us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.690673:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68180 00000100:00100000:1.0:1787826269.691137:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346093184 00000100:00100000:1.0:1787826269.691144:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346093184 00000100:00100000:0.0:1787826269.692797:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6512:1874663346092544:192.168.203.56@tcp:105: 00010000:00010000:0.0:1787826269.692825:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826269.692870:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332509312:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 9832us (12171us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.692879:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21827 00000100:00100000:0.0:1787826269.693626:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23218 00000100:00100000:0.0:1787826269.693637:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346093184:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826269.693746:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346093184:12345-0@lo:1000:mdt_io00_001.0 Request processed in 110us (6430us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826269.693752:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23218 00000100:00100000:3.0:1787826269.695650:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346093184:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826269.696564:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826269.696690:0:24159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332509696 00000100:00100000:0.0:1787826269.696698:0:24159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332509696 00002000:00100000:0.0:1787826269.696748:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000400:0x4c9:0x0] (0->4095) req@ffff9f784f543480 x1874663332509696/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:625/0 lens 488/0 e 0 to 0 dl 1787826280 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826269.696758:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f543480 with x1874663332509696 ext(0->4095) 00002000:00010000:0.0:1787826269.696764:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784f543480 00000100:00100000:0.0:1787826269.696835:0:24159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27629 00000100:00100000:0.0:1787826269.696847:0:24159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332509696:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:0.0:1787826269.697037:0:24159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332509696:12345-192.168.203.56@tcp:3:cat.0 Request processed in 191us (478us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787826269.697044:0:24159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27629 00002000:00100000:0.0:1787826269.697050:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000400:0x4c9:0x0] (0->4095) req@ffff9f784f543480 x1874663332509696/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:625/0 lens 488/4536 e 0 to 0 dl 1787826280 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826269.697058:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f543480 with x1874663332509696 ext(0->4095) 00002000:00010000:0.0:1787826269.697062:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784f543480 00000004:00080000:2.0:1787826269.700733:0:6541:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x240000403:0x1e3f:0x0] is inserted into orphan list, open count = 1 00000040:00080000:2.0:1787826269.700795:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1170, 1240 off1064048 00000040:00080000:2.0:1787826269.700850:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1488, 1320 off1502984 00000004:00080000:2.0:1787826269.700856:0:6541:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7855667000 with version 6772 00000004:00080000:3.0:1787826269.700933:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6772 rpc_version 6772 00000020:00100000:3.0:1787826269.700977:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826269.700987:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b307b80 x1874663346093440/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826269.701014:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346093440:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826269.701079:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826269.701718:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346093440 00000100:00100000:1.0:1787826269.701738:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346093440 00000100:00100000:1.0:1787826269.701929:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23219 00000100:00100000:1.0:1787826269.701950:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346093440:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826269.702329:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78556679c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826269.702443:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346093440:12345-0@lo:1000:osp_up0-1.0 Request processed in 503us (1363us total) trans 8589957297 rc 0/0 00000100:00100000:3.0:1787826269.702976:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346093440:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826269.703997:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23219 00010000:00010000:0.0:1787826269.704660:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704673:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704681:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704687:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826269.704691:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704699:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704716:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:0.0:1787826269.704721:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826269.704724:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704735:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f492ad lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826269.704752:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000403:0x1db3:0x0] 00010000:00010000:0.0:1787826269.704758:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f492a6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704767:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f492a6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704779:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704785:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704791:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704796:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826269.704799:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704805:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704813:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:0.0:1787826269.704815:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826269.704817:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704826:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49259 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826269.704845:0:6541:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001788f689 reply state 0000000055a74064 transno 4294985315 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704856:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704864:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f794b57ab00) 00010000:00010000:0.0:1787826269.704866:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826269.704870:0:6541:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:0.0:1787826269.704873:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00010000:00010000:0.0:1787826269.704877:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704884:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704890:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704980:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826269.704986:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00010000:00010000:0.0:1787826269.704992:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.704999:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.705012:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.705018:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.705025:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.705056:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826269.705067:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.705073:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826269.705076:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826269.705084:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826269.705384:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826269.705476:0:6541:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223356:x1874663332508672:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 137669us (139234us total) trans 4294985315 rc 0/0 00000100:00100000:0.0:1787826269.705484:0:6541:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1222 00010000:00010000:3.0:1787826269.705569:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826269.705575:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.705582:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826269.705585:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826269.705592:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.705616:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.705623:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826269.705644:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 1/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.705651:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f492b4 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f492bb expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.705665:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826269.705667:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826269.705669:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826269.705682:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346093568:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826269.705732:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826269.705783:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346093568 00000100:00100000:3.0:1787826269.705789:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346093568 00000100:00100000:3.0:1787826269.705830:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10974 00000100:00100000:3.0:1787826269.705839:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346093568:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826269.705852:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826269.705868:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f492bb lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f492b4 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826269.705881:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7845cb1400) 00010000:00010000:3.0:1787826269.705888:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826269.705918:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346093568:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (186us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826269.705923:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10974 00000100:00100000:3.0:1787826269.705972:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346093568:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826269.706539:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.706548:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826269.706578:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.706587:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f49275 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4927c expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.706604:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826269.706607:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826269.706608:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826269.706683:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346093696:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826269.706718:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826269.706813:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346093696 00000100:00100000:2.0:1787826269.706819:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346093696 00000100:00100000:2.0:1787826269.706862:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10975 00000100:00100000:2.0:1787826269.706870:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346093696:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826269.706880:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826269.706895:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4927c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49275 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.706907:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7845cb1600) 00010000:00010000:2.0:1787826269.706914:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826269.706960:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346093696:12345-0@lo:103:ldlm_bl.0 Request processed in 92us (241us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826269.706967:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10975 00000100:00100000:3.0:1787826269.707051:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346093696:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826269.716644:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826269.716756:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332509952 00000100:00100000:1.0:1787826269.716767:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332509952 00000100:00100000:1.0:1787826269.716838:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9291 00000100:00100000:1.0:1787826269.716850:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223354:x1874663332509952:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:1.0:1787826269.717076:0:17601:0:(mdd_object.c:4014:mdd_close()) Object [0x240000403:0x1e3f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:1.0:1787826269.717192:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].223, 64 off22464 00000004:00080000:1.0:1787826269.717221:0:17601:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8081929114911ns, next at 8082929097029ns, rc = 0 00010000:00010000:1.0:1787826269.717289:0:17601:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.717305:0:17601:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.717316:0:17601:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.717325:0:17601:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.717331:0:17601:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.717337:0:17601:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826269.717354:0:17601:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826269.717463:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223354:x1874663332509952:12345-192.168.203.56@tcp:35:cat.0 Request processed in 613us (825us total) trans 4294985316 rc 0/0 00000100:00100000:1.0:1787826269.717471:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9291 00010000:00010000:2.0:1787826269.718608:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826269.718619:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.718628:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826269.718633:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.718646:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.718661:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:2.0:1787826269.718663:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826269.718669:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.718675:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f492c9 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17601 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826269.718690:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826269.718692:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826269.718693:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826270.188548:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826270.188570:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346093824:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826270.188642:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826270.188688:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346093952:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826270.188711:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826270.188743:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346093824 00000100:00100000:3.0:1787826270.188750:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346093824 00000100:00100000:3.0:1787826270.188805:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346093952 00000100:00100000:3.0:1787826270.188810:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346093952 00000100:00100000:3.0:1787826270.188825:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6192 00000100:00100000:3.0:1787826270.188833:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346093824:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787826270.188866:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3914715136 avail 3683164160 00000100:00100000:3.0:1787826270.188910:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346093824:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.188917:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6192 00000100:00100000:3.0:1787826270.188950:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6193 00000100:00100000:3.0:1787826270.188958:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346093952:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826270.188969:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662430208 00000100:00100000:3.0:1787826270.188999:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346093952:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (287us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.189005:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6193 00000100:00100000:3.0:1787826270.189047:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346093952:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826270.189076:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346093824:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826270.189228:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346094080:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826270.189359:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826270.189370:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346094208:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826270.189400:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826270.189550:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346094208 00000100:00100000:2.0:1787826270.189556:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346094208 00000100:00100000:2.0:1787826270.189590:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6194 00000100:00100000:2.0:1787826270.189597:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346094208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826270.189608:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662430208 00000100:00100000:2.0:1787826270.189645:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346094208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 48us (244us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.189657:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6194 00000100:00100000:2.0:1787826270.189685:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973832 gen 2 00000004:00080000:2.0:1787826270.189690:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f79465cb100, transno 4294973832 00000100:00100000:2.0:1787826270.189703:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346094208:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826270.189804:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826270.190615:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826270.190628:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346094336:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826270.190665:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826270.191140:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346094080 00000100:00100000:2.0:1787826270.191146:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346094080 00000100:00100000:2.0:1787826270.191184:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346094336 00000100:00100000:2.0:1787826270.191187:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346094336 00000100:00100000:2.0:1787826270.191214:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23220 00000100:00100000:2.0:1787826270.191222:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:3655:x1874663346094080:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826270.191264:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442832384 avail 1296592896 00000100:00100000:2.0:1787826270.191314:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:3655:x1874663346094080:12345-0@lo:41:osp-pre-0-1.0 Request processed in 92us (1509us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.191320:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23220 00000100:00100000:2.0:1787826270.191560:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23221 00000100:00100000:2.0:1787826270.191567:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346094336:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826270.191587:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443442688 avail 1297203200 00000100:00100000:2.0:1787826270.191624:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346094336:12345-0@lo:41:osp-pre-1-0.0 Request processed in 57us (958us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.191630:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23221 00000100:00100000:2.0:1787826270.191673:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346094336:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826270.191969:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346094080:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1787826270.382195:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826270.727460:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826270.727760:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332510080 00000100:00100000:2.0:1787826270.727770:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332510080 00000100:00100000:2.0:1787826270.727856:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68181 00000100:00100000:2.0:1787826270.727868:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223392:x1874663332510080:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826270.727892:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826270.727915:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f492d0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e36 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826270.727940:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1787826270.727993:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f492d7 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.728007:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f492d7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.728022:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f492d7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826270.728138:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3e:0x0]: returning size 4096 00000004:00200000:2.0:1787826270.728144:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787826270.728163:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f492d7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.728191:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f492d0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e36 expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.728386:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f492d7 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e36 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.728395:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:2.0:1787826270.728398:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826270.728402:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826270.728467:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223392:x1874663332510080:12345-192.168.203.56@tcp:101:rm.0 Request processed in 601us (1016us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.728475:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68181 00000100:00100000:3.0:1787826270.731979:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.733030:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332510208 00000100:00100000:0.0:1787826270.733043:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332510208 00000100:00100000:0.0:1787826270.733136:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68182 00000100:00100000:0.0:1787826270.733148:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223392:x1874663332510208:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826270.733171:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826270.733195:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f492de lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e3d expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.733270:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.733288:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.733299:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826270.733345:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18d:0x0]: returning size 4096 00000004:00200000:0.0:1787826270.733350:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826270.733383:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f492de lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.733401:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.733410:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:0.0:1787826270.733412:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826270.733416:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826270.733484:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223392:x1874663332510208:12345-192.168.203.56@tcp:101:rm.0 Request processed in 339us (1516us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.733492:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68182 00000100:00100000:2.0:1787826270.736338:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826270.736375:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332510336 00000100:00100000:2.0:1787826270.736383:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332510336 00000100:00100000:2.0:1787826270.736441:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68183 00000100:00100000:2.0:1787826270.736452:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223392:x1874663332510336:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826270.736473:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826270.736495:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f492ec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e44 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.736551:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.736564:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.736575:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826270.736621:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18b:0x0]: returning size 4096 00000004:00200000:2.0:1787826270.736626:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826270.736659:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f492ec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.736677:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.736686:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7952ca9000) 00010000:00010000:2.0:1787826270.736689:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826270.736692:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826270.736745:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223392:x1874663332510336:12345-192.168.203.56@tcp:101:rm.0 Request processed in 295us (413us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.736751:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68183 00000100:00080000:2.0:1787826270.736777:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826270.742186:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826270.742330:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332510592 00000100:00100000:3.0:1787826270.742340:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332510592 00000100:00100000:3.0:1787826270.742432:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68184 00000100:00100000:3.0:1787826270.742444:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223392:x1874663332510592:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826270.742465:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826270.742490:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f492fa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e4b expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826270.742640:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3e:0x0]: returning size 4096 00010000:00010000:3.0:1787826270.742725:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f492fa lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e4b expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.742740:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:3.0:1787826270.742743:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826270.742748:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f492fa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e4b expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.742766:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826270.742832:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223392:x1874663332510592:12345-192.168.203.56@tcp:101:rm.0 Request processed in 391us (655us total) trans 4294985317 rc 301/301 00000100:00100000:3.0:1787826270.742850:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68184 00000100:00100000:0.0:1787826270.753229:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826270.755988:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332510720 00000100:00100000:1.0:1787826270.756007:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332510720 00000100:00100000:1.0:1787826270.756074:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9292 00000100:00100000:1.0:1787826270.756086:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223392:x1874663332510720:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826270.763382:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223392:x1874663332510720:12345-192.168.203.56@tcp:37:rm.0 Request processed in 7287us (10149us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.763396:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9292 00000100:00100000:1.0:1787826270.779723:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.779857:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332510848 00000100:00100000:0.0:1787826270.779868:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332510848 00000100:00100000:0.0:1787826270.779942:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9293 00000100:00100000:0.0:1787826270.779952:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223392:x1874663332510848:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826270.787293:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223392:x1874663332510848:12345-192.168.203.56@tcp:37:rm.0 Request processed in 7332us (7569us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.787306:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9293 00000100:00100000:2.0:1787826270.799972:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.800303:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332511104 00000100:00100000:0.0:1787826270.800314:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332511104 00000100:00100000:0.0:1787826270.800380:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9294 00000100:00100000:0.0:1787826270.800392:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943004850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223392:x1874663332511104:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:0.0:1787826270.800676:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1171, 344 off1064392 00000040:00080000:0.0:1787826270.800732:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1489, 424 off1503408 00000004:00080000:0.0:1787826270.800739:0:6531:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784548dcc0 with version 6773 00000004:00080000:0.0:1787826270.800757:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6773 rpc_version 6773 00000020:00100000:0.0:1787826270.800799:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826270.800810:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d8bf100 x1874663346094592/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826270.800833:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346094592:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826270.801070:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826270.801142:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7950c25f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.801166:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346094592 00000100:00100000:0.0:1787826270.801171:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346094592 00000100:00100000:0.0:1787826270.801210:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23222 00000100:00100000:0.0:1787826270.801219:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346094592:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826270.801493:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346094592:12345-0@lo:1000:osp_up0-1.0 Request processed in 274us (421us total) trans 8589957298 rc 0/0 00000100:00100000:0.0:1787826270.801501:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23222 00000100:00100000:0.0:1787826270.801562:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346094592:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826270.801650:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943004850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223392:x1874663332511104:12345-192.168.203.56@tcp:35:rm.0 Request processed in 1260us (1685us total) trans 4294985318 rc 0/0 00000100:00100000:0.0:1787826270.801658:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9294 00000100:00100000:1.0:1787826270.812065:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.812158:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332511360 00000100:00100000:0.0:1787826270.812169:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332511360 00000100:00100000:0.0:1787826270.812236:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68185 00000100:00100000:0.0:1787826270.812248:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223392:x1874663332511360:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826270.812271:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826270.812293:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f49308 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.812451:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4930f lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.812465:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4930f lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.812477:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4930f lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826270.812488:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3e:0x0]: returning size 4096 00010000:00010000:0.0:1787826270.812588:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f49308 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e52 expref: 16 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.812612:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4930f lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e52 expref: 15 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.812621:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:0.0:1787826270.812623:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787826270.812626:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826270.812677:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:223392:x1874663332511360:12345-192.168.203.56@tcp:101:rm.0 Request processed in 431us (622us total) trans 4294985319 rc 0/0 00000100:00100000:0.0:1787826270.812683:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68185 00000100:00100000:2.0:1787826270.842629:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826270.846534:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332511488 00000100:00100000:3.0:1787826270.846552:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332511488 00000100:00100000:2.0:1787826270.848342:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68186 00000100:00100000:2.0:1787826270.848357:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223392:x1874663332511488:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:2.0:1787826270.848408:0:35092:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345594368 00000100:00100000:2.0:1787826270.848548:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223392:x1874663332511488:12345-192.168.203.56@tcp:41:rm.0 Request processed in 194us (5926us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.848558:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68186 00000100:00100000:1.0:1787826270.854991:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826270.858057:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332511616 00000100:00100000:3.0:1787826270.858071:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332511616 00000100:00100000:3.0:1787826270.858149:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68187 00000100:00100000:3.0:1787826270.858161:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:223392:x1874663332511616:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826270.858186:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826270.858445:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4931d lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e59 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858508:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858526:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858536:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858546:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858554:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858559:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858617:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f49332 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858626:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f49332 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858633:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f49332 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826270.858652:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f49332 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858665:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858672:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858682:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858689:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858694:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826270.858699:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858706:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858716:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:3.0:1787826270.858718:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826270.858723:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858729:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49324 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858740:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858746:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858764:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858769:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826270.858773:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858779:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858791:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f794b57ab00) 00010000:00010000:3.0:1787826270.858793:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826270.858796:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858802:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4932b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858824:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4931d lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e59 expref: 17 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858835:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f49332 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e59 expref: 16 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.858842:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:3.0:1787826270.858845:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826270.858847:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826270.858912:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:223392:x1874663332511616:12345-192.168.203.56@tcp:101:rm.0 Request processed in 754us (3930us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.858919:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68187 00000100:00100000:1.0:1787826270.867960:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826270.868532:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332511744 00000100:00100000:3.0:1787826270.868543:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332511744 00000100:00100000:3.0:1787826270.868593:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68188 00000100:00100000:3.0:1787826270.868601:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223392:x1874663332511744:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826270.868621:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826270.868641:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49339 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e60 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.868697:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49340 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.868710:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49340 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.868723:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49340 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826270.869301:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db3:0x0]: returning size 6 00000004:00200000:3.0:1787826270.869306:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:3.0:1787826270.869347:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49339 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e60 expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.869363:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49340 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e60 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.869373:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:3.0:1787826270.869376:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826270.869379:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826270.870885:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223392:x1874663332511744:12345-192.168.203.56@tcp:101:rm.0 Request processed in 2276us (2928us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.870896:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68188 00000100:00100000:0.0:1787826270.882102:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.882168:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332512128 00000100:00100000:0.0:1787826270.882178:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332512128 00000100:00100000:0.0:1787826270.882229:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68189 00000100:00100000:0.0:1787826270.882238:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223392:x1874663332512128:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826270.882290:0:24703:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826270.882321:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49340 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e60 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.882336:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826270.882342:0:24703:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826270.882416:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346094720:0@lo:1000:mdt00_007.0 00000100:00100000:0.0:1787826270.882467:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826270.882494:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826270.883283:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346094720 00000100:00100000:1.0:1787826270.883291:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346094720 00000100:00100000:1.0:1787826270.883335:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23223 00000100:00100000:1.0:1787826270.883343:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:24703:x1874663346094720:12345-0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826270.884405:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:24703:x1874663346094720:12345-0@lo:1000:mdt00_007.0 Request processed in 849us (1723us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.884424:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23223 00000100:00100000:1.0:1787826270.884989:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826270.885034:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332511872 00000100:00100000:1.0:1787826270.885043:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332511872 00000100:00100000:1.0:1787826270.885134:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10976 00000100:00100000:1.0:1787826270.885145:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:2394:x1874663332511872:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:1.0:1787826270.885165:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.886140:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f49332 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e59 expref: 16 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.886171:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:1.0:1787826270.886191:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826270.887004:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:2394:x1874663332511872:12345-192.168.203.56@tcp:103:rm.0 Request processed in 1855us (2013us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.887015:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10976 00000100:00100000:1.0:1787826270.887083:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346094720:0@lo:1000:mdt00_007.0 00010000:00010000:1.0:1787826270.887390:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.887400:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.887411:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.887431:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346094848:0@lo:101:mdt00_007.0 00000100:00100000:1.0:1787826270.890058:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.890118:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.890192:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346094848 00000100:00100000:0.0:1787826270.890210:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346094848 00000100:00100000:0.0:1787826270.890277:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23224 00000100:00100000:0.0:1787826270.890290:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:24703:x1874663346094848:12345-0@lo:101:mdt00_007.0 00010000:00010000:0.0:1787826270.890310:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826270.890331:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49347 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.890361:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.890372:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 4/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49347 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.890407:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.890420:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 14 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.890434:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346094976:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826270.890506:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.896072:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346094976:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826270.896138:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49347 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.896152:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:0.0:1787826270.896155:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826270.896160:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49347 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.896169:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826270.896224:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346094848:12345-0@lo:101:mdt00_007.0 Request processed in 5938us (6172us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.896232:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23224 00000100:00100000:1.0:1787826270.896377:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346094848:0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826270.896395:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.896406:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.897698:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826270.897777:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332512256 00000100:00100000:1.0:1787826270.897850:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332512256 00010000:00010000:1.0:1787826270.897896:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 15 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.897927:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21828 00000100:00100000:1.0:1787826270.897947:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:2394:x1874663332512256:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826270.897962:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.897969:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 15 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.897981:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 15 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.898001:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f492e5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e3d expref: 15 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.898014:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:1.0:1787826270.898020:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49347 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.898031:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49347 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.898040:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49347 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.898073:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49347 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.898089:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6513:1874663346095104:0@lo:105: 00000100:00100000:1.0:1787826270.898165:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.898208:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826270.898329:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346095104 00000100:00100000:2.0:1787826270.898335:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346095104 00000100:00100000:2.0:1787826270.898371:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7798 00000100:00100000:2.0:1787826270.898378:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6513:x1874663346095104:12345-0@lo:105: 00010000:00010000:2.0:1787826270.898389:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.898399:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.898408:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.898416:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.898427:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.898435:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.898467:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece200) 00000100:00100000:2.0:1787826270.898510:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6513:x1874663346095104:12345-0@lo:105: Request processed in 132us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.898517:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7798 00000100:00100000:1.0:1787826270.898884:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6513:1874663346095104:0@lo:105: 00010000:00010000:1.0:1787826270.898903:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826270.898959:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:2394:x1874663332512256:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1011us (1265us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.898968:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21828 00010000:00010000:1.0:1787826270.899002:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.899011:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826270.899035:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18d:0x0] 00010000:00010000:1.0:1787826270.899073:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.899080:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.899090:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.899132:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346095232:0@lo:101:mdt00_007.0 00000100:00100000:1.0:1787826270.899169:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.899200:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826270.899340:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346095232 00000100:00100000:2.0:1787826270.899346:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346095232 00000100:00100000:2.0:1787826270.899382:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23225 00000100:00100000:2.0:1787826270.899390:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346095232:12345-0@lo:101:mdt00_007.0 00010000:00010000:2.0:1787826270.899404:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826270.899419:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49355 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.899438:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 2/0,1 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.899824:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 4/0,0 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49355 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.899843:0:8152:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:00080000:2.0:1787826270.900125:0:8152:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00010000:00010000:2.0:1787826270.900170:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 3/0,0 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49355 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.900178:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f794b57ab00) 00010000:00010000:2.0:1787826270.900182:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826270.900186:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 3/0,0 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49355 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.900193:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826270.900603:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:24703:x1874663346095232:12345-0@lo:101:mdt00_007.0 Request processed in 1211us (1431us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.900613:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23225 00000100:00100000:1.0:1787826270.901829:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346095232:0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826270.901862:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.901874:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.903710:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826270.903729:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.903792:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.903800:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826270.903831:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f49283 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4928a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.903860:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f492a6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826270.903868:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f492a6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.903885:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f492a6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.903891:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826270.903898:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4929f lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f492a6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826270.903910:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985319 is committed 00000004:00080000:1.0:1787826270.903937:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 360 committed 00000100:00100000:2.0:1787826270.903952:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346095360:0@lo:103:jbd2/dm-1-8.0 00000001:00080000:1.0:1787826270.903958:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985315 is committed 00000001:00080000:1.0:1787826270.903967:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985314 is committed 00000020:00080000:1.0:1787826270.903978:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3385 batchid updated 00000100:00100000:1.0:1787826270.904002:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346095488:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787826270.904027:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826270.904091:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346095360 00000100:00100000:2.0:1787826270.904098:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346095360 00010000:00010000:1.0:1787826270.904135:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.904145:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.904153:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.904160:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826270.904166:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.904174:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826270.904174:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10977 00000100:00100000:2.0:1787826270.904184:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346095360:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826270.904192:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f794b57ab00) 00010000:00010000:1.0:1787826270.904197:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826270.904202:0:3638:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49355 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.904206:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.904217:0:3638:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49355 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.904230:0:3638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49355 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.904230:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4928a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49283 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.904259:0:3638:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49355 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.904275:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0001_UUID:3638:1874663346095744:0@lo:105: 00000100:00100000:1.0:1787826270.904302:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.904330:0:3638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b13c0 going to sleep for 11 seconds 00000020:00080000:1.0:1787826270.904342:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3385 00010000:00010000:2.0:1787826270.904361:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x4ec94fd6 (ffff9f7845cb1100) 00010000:00010000:2.0:1787826270.904370:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:1.0:1787826270.904396:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d9b1b40 with version 6774 00000020:00080000:1.0:1787826270.904409:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3382 cancel update log [0x200000401:0x3:0x0].1484: rc = 0 00000100:00100000:2.0:1787826270.904412:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346095360:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 228us (384us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.904421:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10977 00000100:00100000:2.0:1787826270.904537:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346095360:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:1.0:1787826270.905603:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3382 cancel update log [0x240000400:0x3:0x0].1166: rc = 0 00000004:00080000:1.0:1787826270.905638:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d9b19c0 with version 6775 00000020:00080000:1.0:1787826270.905643:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3383 cancel update log [0x200000401:0x3:0x0].1485: rc = 0 00000020:00080000:1.0:1787826270.905664:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3383 cancel update log [0x240000400:0x3:0x0].1167: rc = 0 00000004:00080000:1.0:1787826270.905682:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d9b1900 with version 6776 00000020:00080000:1.0:1787826270.905685:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3384 cancel update log [0x200000401:0x3:0x0].1486: rc = 0 00000020:00080000:1.0:1787826270.905702:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3384 cancel update log [0x240000400:0x3:0x0].1168: rc = 0 00000004:00080000:1.0:1787826270.905719:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d9b10c0 with version 6777 00000020:00080000:1.0:1787826270.905722:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3385 cancel update log [0x200000401:0x3:0x0].1487: rc = 0 00000020:00080000:1.0:1787826270.905737:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3385 cancel update log [0x240000400:0x3:0x0].1169: rc = 0 00000020:00080000:1.0:1787826270.905744:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3385 00000100:00100000:1.0:1787826270.905773:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346095616:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826270.905962:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.906026:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346095744 00000100:00100000:1.0:1787826270.906032:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346095744 00000100:00100000:1.0:1787826270.906084:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7799 00000100:00100000:1.0:1787826270.906094:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:3638:x1874663346095744:12345-0@lo:105: 00010000:00010000:1.0:1787826270.906130:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.906142:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.906151:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.906159:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 5/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.906167:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 6/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.906175:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 6/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.906198:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece600) 00000100:00100000:1.0:1787826270.906236:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:3638:x1874663346095744:12345-0@lo:105: Request processed in 144us (1933us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.906243:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7799 00000004:00080000:1.0:1787826270.906263:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6774 rpc_version 6774 00000020:00100000:1.0:1787826270.906285:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826270.906293:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977e0ed80 x1874663346096000/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826270.906308:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0ed80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346096000:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826270.906340:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.906363:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f592000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826270.906397:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.906812:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0001_UUID:3638:1874663346095744:0@lo:105: 00010000:00010000:1.0:1787826270.906833:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.906842:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49260 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826270.907131:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346095616 00000100:00100000:3.0:1787826270.907137:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346095616 00000100:00100000:3.0:1787826270.907177:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30772 00000100:00100000:3.0:1787826270.907185:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346095616:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:3.0:1787826270.907209:0:24218:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1225 count 1 00010000:00010000:3.0:1787826270.907231:0:24218:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.907250:0:24218:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.907259:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=2s v=(1 0) 00010000:00010000:3.0:1787826270.907264:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826270.907269:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.907294:0:24218:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.907307:0:24218:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 5 pid: 8443 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826270.907319:0:24218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:24218:1874663346096128:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826270.907390:0:24218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.908055:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346096000 00000100:00100000:0.0:1787826270.908062:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346096000 00000100:00100000:0.0:1787826270.908205:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23226 00000100:00100000:0.0:1787826270.908215:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346096000:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826270.908381:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346096000:12345-0@lo:1000:osp_up0-1.0 Request processed in 167us (2038us total) trans 8589957299 rc 0/0 00000100:00100000:0.0:1787826270.908393:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23226 00000100:00100000:1.0:1787826270.908591:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346095488 00000100:00100000:1.0:1787826270.908596:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346095488 00000100:00100000:1.0:1787826270.908642:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10978 00000100:00100000:1.0:1787826270.908650:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346095488:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826270.908661:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.908684:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f492a6 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4929f expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.908699:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:1.0:1787826270.908708:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826270.908745:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+10:3655:x1874663346095488:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 97us (2347us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.908764:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10978 00000100:00100000:1.0:1787826270.909059:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346095488:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826270.909137:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0ed80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346096000:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826270.909149:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6775 rpc_version 6775 00000020:00100000:1.0:1787826270.909167:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826270.909174:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977e0e300 x1874663346096384/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826270.909194:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346096384:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826270.909231:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.909643:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d58840 going to sleep for 11 seconds 00010000:00010000:1.0:1787826270.909663:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.909673:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826270.909697:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18d:0x0] 00000100:00100000:1.0:1787826270.909731:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346096512:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826270.909941:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826270.911476:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346096384 00000100:00100000:3.0:1787826270.911490:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346096384 00000100:00100000:3.0:1787826270.911634:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826270.911924:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332512384 00000100:00100000:3.0:1787826270.911988:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332512384 00010000:00010000:3.0:1787826270.912044:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 6 pid: 8443 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826270.912077:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21829 00000100:00100000:3.0:1787826270.912087:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7947dfc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332512384:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826270.912099:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826270.912103:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 6 pid: 8443 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.912134:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 6 pid: 8443 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.912156:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826270.912160:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826270.912164:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.912171:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.912188:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826270.912522:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dfc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332512384:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 434us (888us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.912531:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21829 00000100:00100000:2.0:1787826270.912586:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23227 00000100:00080000:3.0:1787826270.912593:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826270.912599:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346096384:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826270.912965:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346096384:12345-0@lo:1000:osp_up0-1.0 Request processed in 370us (3733us total) trans 8589957300 rc 0/0 00000100:00100000:2.0:1787826270.912974:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23227 00010000:00010000:3.0:1787826270.913136:0:24218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f49298 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e28 expref: 5 pid: 8443 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826270.913158:0:24218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:24218:1874663346096128:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826270.913182:0:24218:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.913192:0:24218:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.913313:0:24218:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.913325:0:24218:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.913333:0:24218:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.913359:0:24218:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787826270.913522:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1787826270.913529:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346095616:12345-0@lo:6:osp-syn-0-1.0 Request processed in 6343us (7565us total) trans 4294973834 rc 0/0 00010000:00010000:2.0:1787826270.913531:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826270.913542:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30772 00010000:00010000:2.0:1787826270.913548:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826270.913555:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.913567:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.913586:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826270.913590:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7977e0f480/2, rc 0, transno 4294973834 00000100:00100000:3.0:1787826270.913600:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346095616:0@lo:6:osp-syn-0-1.0 00010000:00010000:2.0:1787826270.913601:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4aeb212/0xe21309aa81f49363 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.913626:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826270.913629:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826270.913631:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826270.914354:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346096384:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826270.914367:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6776 rpc_version 6776 00000020:00100000:2.0:1787826270.914390:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826270.914396:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784f542d80 x1874663346096768/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826270.914417:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346096768:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826270.914574:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b13c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826270.914577:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346096512 00000100:00100000:3.0:1787826270.914585:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346096512 00000100:00100000:3.0:1787826270.914632:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23228 00000100:00100000:3.0:1787826270.914642:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346096512:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826270.914737:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346096512:12345-0@lo:1000:mdt00_007.0 Request processed in 96us (4795us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.914745:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23228 00000100:00100000:2.0:1787826270.914897:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826270.914927:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826270.914981:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346096768 00000100:00100000:1.0:1787826270.914986:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346096768 00000100:00100000:1.0:1787826270.915014:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23229 00000100:00100000:1.0:1787826270.915023:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346096768:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826270.915168:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346096768:12345-0@lo:1000:osp_up0-1.0 Request processed in 147us (271us total) trans 8589957301 rc 0/0 00000100:00100000:1.0:1787826270.915176:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23229 00000100:00100000:1.0:1787826270.915209:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346096768:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826270.915218:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6777 rpc_version 6777 00000020:00100000:1.0:1787826270.915231:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826270.915236:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977c6f100 x1874663346097024/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826270.915255:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346097024:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826270.915281:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.915295:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826270.915314:0:24703:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985314 gen 2 00000004:00080000:1.0:1787826270.915319:0:24703:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985314 committed transno 4294985314 00000100:00100000:1.0:1787826270.915340:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346096512:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826270.915387:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346097152:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826270.915412:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826270.915921:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346097024 00000100:00100000:2.0:1787826270.915936:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346097024 00000100:00100000:2.0:1787826270.915962:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23230 00000100:00100000:2.0:1787826270.915970:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346097024:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826270.916079:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346097024:12345-0@lo:1000:osp_up0-1.0 Request processed in 109us (796us total) trans 8589957302 rc 0/0 00000100:00100000:2.0:1787826270.916086:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23230 00000100:00100000:2.0:1787826270.916100:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346097152 00000100:00100000:2.0:1787826270.916120:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346097152 00000100:00100000:2.0:1787826270.916136:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23231 00000100:00100000:2.0:1787826270.916141:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346097152:12345-0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826270.916196:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346097152:12345-0@lo:1000:mdt00_007.0 Request processed in 54us (783us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.916202:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23231 00000100:00100000:1.0:1787826270.916916:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346097024:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826270.916945:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826270.916957:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346097152:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826270.917015:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346097280:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826270.917055:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.917087:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826270.917239:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346097280 00000100:00100000:3.0:1787826270.917245:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346097280 00000100:00100000:3.0:1787826270.917282:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23232 00000100:00100000:3.0:1787826270.917290:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346097280:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826270.917383:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346097280:12345-0@lo:1000:mdt00_007.0 Request processed in 93us (327us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826270.917390:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23232 00000100:00100000:1.0:1787826270.917516:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346097280:0@lo:1000:mdt00_007.0 00010000:00010000:1.0:1787826270.917567:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.917576:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.917585:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.917597:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346097408:0@lo:101:mdt00_007.0 00000100:00100000:1.0:1787826270.917622:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.917645:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e813c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.917719:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346097408 00000100:00100000:0.0:1787826270.917723:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346097408 00000100:00100000:0.0:1787826270.917765:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23233 00000100:00100000:0.0:1787826270.917772:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346097408:12345-0@lo:101:mdt00_007.0 00010000:00010000:0.0:1787826270.917939:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826270.917960:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49371 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4936a expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.917983:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49371 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4936a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.917996:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49371 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4936a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.918003:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:0.0:1787826270.918006:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826270.918009:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826270.918050:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:24703:x1874663346097408:12345-0@lo:101:mdt00_007.0 Request processed in 278us (427us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.918056:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23233 00000100:00100000:1.0:1787826270.918190:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346097408:0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826270.918203:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.918215:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.918239:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.918247:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.918254:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787826270.918617:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].209, 64 off21568 00000004:00080000:1.0:1787826270.918630:0:24703:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8083130523331ns, next at 8084130520365ns, rc = 0 00000040:00080000:1.0:1787826270.918668:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].928, 384 off872672 00000040:00080000:1.0:1787826270.918715:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].931, 464 off971256 00000004:00080000:1.0:1787826270.918721:0:24703:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7844e813c0 with version 1811 00000004:00080000:0.0:1787826270.919026:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1811 rpc_version 1811 00000020:00100000:0.0:1787826270.919051:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826270.919060:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d8bfb80 x1874663346097664/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1664/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826270.919080:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346097664:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826270.919159:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826270.919182:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826270.919239:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346097664 00000100:00100000:3.0:1787826270.919245:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346097664 00000100:00100000:3.0:1787826270.919284:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23234 00000100:00100000:3.0:1787826270.919293:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346097664:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826270.919628:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346097664:12345-0@lo:1000:osp_up1-0.0 Request processed in 334us (466us total) trans 4294985320 rc 0/0 00000100:00100000:3.0:1787826270.919635:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23234 00010000:00010000:0.0:1787826270.919889:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.919903:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.919913:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.919921:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.919937:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.919944:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.919964:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826270.919982:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.919988:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.919994:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920000:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826270.920005:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920013:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920023:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f7845cb1000) 00010000:00010000:0.0:1787826270.920026:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826270.920029:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920036:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49378 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920047:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920055:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920062:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920076:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787826270.920086:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18d:0x0] 00010000:00010000:0.0:1787826270.920093:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826270.920098:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920126:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826270.920130:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920140:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920147:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f7845cb1000) 00010000:00010000:0.0:1787826270.920150:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826270.920153:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920159:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4937f lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db3:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920171:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826270.920175:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920181:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826270.920185:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826270.920193:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920215:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920221:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826270.920239:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 1/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920251:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4936a lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49371 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920263:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826270.920265:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826270.920267:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826270.920277:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920284:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920291:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920302:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826270.920306:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18d:0x0] 00010000:00010000:0.0:1787826270.920310:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.920317:0:24703:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.920376:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223392:x1874663332512128:12345-192.168.203.56@tcp:36:rm.0 Request processed in 38137us (38282us total) trans 8589957303 rc 0/0 00000100:00100000:0.0:1787826270.920383:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68189 00000100:00100000:1.0:1787826270.920496:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346097664:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787826270.920514:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826270.920519:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.920526:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826270.920531:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826270.920538:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.920556:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.920562:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826270.920581:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.920589:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece200/0xe21309aa81f49347 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4934e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.920601:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826270.920603:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826270.920604:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826270.920613:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346097792:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826270.920646:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.920677:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826270.920684:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346097920:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826270.920702:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.920722:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346097792 00000100:00100000:1.0:1787826270.920727:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346097792 00010000:00010000:3.0:1787826270.920911:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826270.921067:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346097920 00000100:00100000:1.0:1787826270.921071:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346097920 00000100:00100000:3.0:1787826270.921732:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10979 00000100:00100000:3.0:1787826270.921744:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+10:3653:x1874663346097792:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826270.921768:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826270.921792:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49371 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db3:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4936a expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.921808:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db3:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:3.0:1787826270.921818:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826270.921869:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346097792:12345-0@lo:103:ldlm_bl.0 Request processed in 129us (1222us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.921876:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10979 00000100:00100000:3.0:1787826270.921893:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10980 00000100:00100000:3.0:1787826270.921899:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+8:3655:x1874663346097920:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826270.921906:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826270.921916:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4934e lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49347 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.921925:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:3.0:1787826270.921942:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826270.921974:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346097920:12345-0@lo:103:ldlm_bl.0 Request processed in 75us (1271us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.921980:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10980 00000100:00100000:2.0:1787826270.922097:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346097920:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826270.922204:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346097792:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826270.931641:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826270.933562:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332512768 00000100:00100000:2.0:1787826270.933578:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332512768 00000100:00100000:2.0:1787826270.933699:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9295 00000100:00100000:2.0:1787826270.933710:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223392:x1874663332512768:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:2.0:1787826270.933971:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223392:x1874663332512768:12345-192.168.203.56@tcp:35:rm.0 Request processed in 261us (2337us total) trans 4294985321 rc 0/0 00000100:00100000:2.0:1787826270.933979:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9295 00000100:00100000:3.0:1787826270.938721:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826270.938776:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.939034:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332513024 00000100:00100000:0.0:1787826270.939044:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332513024 00000100:00100000:0.0:1787826270.939135:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68190 00000100:00100000:0.0:1787826270.939157:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223392:x1874663332513024:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826270.939218:0:6529:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826270.939246:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4930f lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e52 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.939270:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f492d7 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e36 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.939284:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f491b1 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800de2 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.939293:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:0.0:1787826270.939298:0:6529:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826270.939341:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.939350:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.939360:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.939383:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098048:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826270.939446:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826270.939476:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ed80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826270.939682:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346098048 00000100:00100000:1.0:1787826270.939687:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346098048 00000100:00100000:1.0:1787826270.939729:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23235 00000100:00100000:1.0:1787826270.939737:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:6529:x1874663346098048:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826270.939759:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826270.939774:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49386 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.939806:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.939814:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.939822:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49386 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.939846:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.939859:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 7 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.939870:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346098176:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826270.939950:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.939960:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 9 pid: 24703 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.939975:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346098304:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826270.939999:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826270.940583:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332512512 00000100:00100000:2.0:1787826270.940590:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332512512 00000100:00100000:2.0:1787826270.940681:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10981 00000100:00100000:2.0:1787826270.940690:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332512512:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826270.940701:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826270.940718:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f491a3 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ddb expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.940732:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:2.0:1787826270.940736:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826270.940792:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332512512:12345-192.168.203.56@tcp:103:rm.0 Request processed in 103us (2080us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.940799:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10981 00000100:00100000:1.0:1787826270.945423:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346098304:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826270.949959:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826270.950005:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.950772:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332513152 00000100:00100000:0.0:1787826270.950852:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332513152 00010000:00010000:0.0:1787826270.950897:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 9 pid: 24703 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.950945:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332513280 00000100:00100000:0.0:1787826270.950948:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332513280 00010000:00010000:0.0:1787826270.950971:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 8 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.950996:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21830 00000100:00100000:0.0:1787826270.951007:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332513152:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826270.951023:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826270.951030:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 9 pid: 24703 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.951043:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 9 pid: 24703 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.951064:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49187 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dd4 expref: 9 pid: 24703 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.951080:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826270.951086:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49386 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.951095:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826270.951288:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21831 00000100:00100000:2.0:1787826270.951300:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332513280:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826270.951321:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826270.951327:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 8 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.951339:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 8 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.951352:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826270.951357:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49386 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.951366:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49386 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.951376:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49386 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.951407:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49386 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826270.951420:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f540000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346098432:0@lo:105: 00000100:00100000:0.0:1787826270.951558:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332513152:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 552us (1608us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.951567:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21830 00010000:00010000:1.0:1787826270.951756:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f491f0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dfe expref: 8 pid: 24702 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.951769:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346098176:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826270.951806:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49386 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.951813:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826270.951816:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826270.951818:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00080000:3.0:1787826270.951922:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826270.952346:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826270.952417:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.953067:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346098432 00000100:00100000:0.0:1787826270.953076:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346098432 00000100:00100000:0.0:1787826270.953162:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7800 00000100:00100000:0.0:1787826270.953179:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346098432:12345-0@lo:105: 00010000:00010000:0.0:1787826270.953194:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.953203:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.953213:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.953221:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.953230:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.953238:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.953256:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43800) 00000100:00100000:0.0:1787826270.953310:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346098432:12345-0@lo:105: Request processed in 140us (964us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.953317:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7800 00000100:00100000:0.0:1787826270.953364:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:6529:x1874663346098048:12345-0@lo:101:mdt00_002.0 Request processed in 13627us (13916us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.953371:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23235 00000100:00100000:0.0:1787826270.953416:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098048:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787826270.953444:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826270.953467:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826270.953512:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.953520:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.953530:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.953548:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098560:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826270.953582:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826270.953603:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826270.953748:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f540000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346098432:0@lo:105: 00010000:00010000:2.0:1787826270.953774:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826270.953820:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332513280:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2523us (3812us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826270.953828:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21831 00000100:00100000:0.0:1787826270.955295:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346098560 00000100:00100000:0.0:1787826270.955307:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346098560 00000100:00100000:0.0:1787826270.955372:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23236 00000100:00100000:0.0:1787826270.955381:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:6529:x1874663346098560:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787826270.955402:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826270.955424:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4939b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49394 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.955459:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4939b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49394 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.955474:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4939b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49394 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.955483:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a500) 00010000:00010000:0.0:1787826270.955485:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826270.955489:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:0.0:1787826270.955561:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:6529:x1874663346098560:12345-0@lo:101:mdt00_002.0 Request processed in 181us (1978us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.955571:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23236 00000100:00100000:2.0:1787826270.955679:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098560:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787826270.955706:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.955717:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826270.955741:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826270.955805:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ded80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098688:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826270.955841:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826270.955867:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.956032:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346098688 00000100:00100000:0.0:1787826270.956041:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346098688 00000100:00100000:0.0:1787826270.956073:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23237 00000100:00100000:0.0:1787826270.956080:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:6529:x1874663346098688:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826270.956294:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:6529:x1874663346098688:12345-0@lo:1000:mdt00_002.0 Request processed in 214us (451us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.956302:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23237 00000100:00100000:0.0:1787826270.956340:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ded80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098688:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826270.956387:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098816:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826270.956413:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826270.956489:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.956507:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346098816 00000100:00100000:0.0:1787826270.956511:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346098816 00000100:00100000:0.0:1787826270.956534:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23238 00000100:00100000:0.0:1787826270.956540:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:6529:x1874663346098816:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826270.956629:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:6529:x1874663346098816:12345-0@lo:1000:mdt00_002.0 Request processed in 88us (214us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826270.956635:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23238 00000100:00100000:0.0:1787826270.956890:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098816:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1787826270.956957:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.956965:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.956973:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.956985:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098944:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826270.957008:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826270.957131:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826270.957802:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346098944 00000100:00100000:1.0:1787826270.957810:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346098944 00000100:00100000:1.0:1787826270.957997:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23239 00000100:00100000:1.0:1787826270.958008:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:6529:x1874663346098944:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826270.958021:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826270.958039:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f493a2 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.958065:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.958075:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f493a2 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.958099:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.958195:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 6 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.958208:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346099072:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826270.958573:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826270.966705:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.966915:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346099072:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826270.966960:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493a2 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.966974:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:0.0:1787826270.966977:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826270.966980:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493a2 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.966990:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:0.0:1787826270.967060:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+16:6529:x1874663346098944:12345-0@lo:101:mdt00_002.0 Request processed in 9054us (10048us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.967071:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23239 00000100:00100000:1.0:1787826270.967153:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332513408 00000100:00100000:1.0:1787826270.967161:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332513408 00010000:00010000:1.0:1787826270.967200:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 7 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.967234:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21832 00000100:00100000:1.0:1787826270.967244:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332513408:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826270.967275:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.967282:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 7 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.967294:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 7 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.967318:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4920c lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e05 expref: 7 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.967330:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:1.0:1787826270.967335:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493a2 expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.967345:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f493a2 expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.967353:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f493a2 expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.967381:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493a2 expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.967396:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346099200:0@lo:105: 00000100:00100000:1.0:1787826270.967433:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.967470:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826270.967661:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346098944:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787826270.967679:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826270.967689:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.967824:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346099200 00000100:00100000:0.0:1787826270.967830:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346099200 00000100:00100000:0.0:1787826270.967995:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7801 00000100:00100000:0.0:1787826270.968004:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346099200:12345-0@lo:105: 00010000:00010000:0.0:1787826270.968013:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.968021:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.968028:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.968034:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.968041:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.968048:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.968073:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42e00) 00010000:00010000:1.0:1787826270.968223:0:6529:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968231:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.968357:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346099200:12345-0@lo:105: Request processed in 354us (922us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.968365:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7801 00000100:00100000:3.0:1787826270.968484:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346099200:0@lo:105: 00010000:00010000:3.0:1787826270.968496:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826270.968543:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332513408:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1300us (1847us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826270.968551:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21832 00010000:00010000:1.0:1787826270.968589:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968602:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968611:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968639:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968647:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968653:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968680:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968687:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.968696:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.968712:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346099328:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787826270.968744:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.968780:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.969210:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346099328 00000100:00100000:0.0:1787826270.969216:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346099328 00000100:00100000:0.0:1787826270.969258:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23240 00000100:00100000:0.0:1787826270.969266:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+16:6529:x1874663346099328:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787826270.969405:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826270.969421:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f493be expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.969441:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.969450:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 4/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.969475:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.969487:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 7 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.969499:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d817480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346099456:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826270.969556:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ed80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826270.974098:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826270.974265:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d817480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346099456:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826270.974359:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.974491:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332513536 00000100:00100000:1.0:1787826270.974507:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332513536 00010000:00010000:0.0:1787826270.974528:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7952ca9000) 00010000:00010000:0.0:1787826270.974532:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826270.974536:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.974542:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 8 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.974547:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.974554:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:1.0:1787826270.974618:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21833 00000100:00100000:0.0:1787826270.974619:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+17:6529:x1874663346099328:12345-0@lo:101:mdt00_002.0 Request processed in 5353us (5874us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.974626:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23240 00000100:00100000:1.0:1787826270.974628:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c82050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332513536:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826270.974646:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.974653:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 8 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.974665:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 8 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.974691:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f492f3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e44 expref: 8 pid: 6529 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.974706:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7952ca9000) 00010000:00010000:1.0:1787826270.974711:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 16 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.974721:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 16 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.974733:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 16 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.974763:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 16 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826270.974777:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346099584:0@lo:105: 00000100:00100000:1.0:1787826270.974822:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.974864:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81b40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826270.974871:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346099328:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787826270.974887:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 4/0,1 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.974897:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 4/0,1 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.974970:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346099584 00000100:00100000:0.0:1787826270.974975:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346099584 00000100:00100000:0.0:1787826270.975012:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7802 00000100:00100000:0.0:1787826270.975020:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346099584:12345-0@lo:105: 00010000:00010000:0.0:1787826270.975029:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 5/0,1 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.975037:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 5/0,1 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.975045:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 5/0,1 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.975052:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.975058:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.975067:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.975091:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece400) 00010000:00010000:3.0:1787826270.975612:0:6529:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826270.975624:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.976418:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346099584:12345-0@lo:105: Request processed in 1393us (1590us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.976428:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7802 00000100:00100000:1.0:1787826270.976447:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346099584:0@lo:105: 00010000:00010000:1.0:1787826270.976463:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826270.976519:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c82050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332513536:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1890us (2429us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.976528:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21833 00000100:00100000:3.0:1787826270.976817:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346099712:0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787826270.977435:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826270.977510:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826270.977556:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346099712 00000100:00100000:0.0:1787826270.977563:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346099712 00000100:00100000:0.0:1787826270.977605:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23241 00000100:00100000:0.0:1787826270.977614:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:6529:x1874663346099712:12345-0@lo:403:mdt00_002.0 00000100:00100000:0.0:1787826270.980146:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:6529:x1874663346099712:12345-0@lo:403:mdt00_002.0 Request processed in 2494us (2672us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826270.980159:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23241 00000100:00100000:3.0:1787826270.980363:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346099712:0@lo:403:mdt00_002.0 00000040:00080000:0.0:1787826270.981492:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1172, 952 off1065344 00000040:00080000:0.0:1787826270.981654:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1490, 1032 off1504440 00000004:00080000:0.0:1787826270.981660:0:6529:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7848d5ed80 with version 6778 00000004:00080000:1.0:1787826270.982317:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6778 rpc_version 6778 00000020:00100000:1.0:1787826270.982353:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826270.982363:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7850ad7800 x1874663346099968/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826270.982387:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346099968:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826270.982444:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.982484:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826270.982918:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346099968 00000100:00100000:2.0:1787826270.982937:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346099968 00000100:00100000:2.0:1787826270.982994:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23242 00000100:00100000:2.0:1787826270.983003:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:7471:x1874663346099968:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826270.983672:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+18:7471:x1874663346099968:12345-0@lo:1000:osp_up0-1.0 Request processed in 668us (1226us total) trans 8589957304 rc 0/0 00000100:00100000:2.0:1787826270.983681:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23242 00000100:00100000:1.0:1787826270.983836:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346099968:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787826270.983893:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e3e:0x0]: returning size 4096 00010000:00010000:0.0:1787826270.983910:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.983917:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.983922:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.983926:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826270.983941:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.983949:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.983967:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:0.0:1787826270.983970:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826270.983974:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.983980:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f493b7 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984061:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984071:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984078:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984101:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826270.984135:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18b:0x0] 00010000:00010000:0.0:1787826270.984143:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984150:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984156:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984161:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826270.984164:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984170:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984178:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:0.0:1787826270.984181:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826270.984184:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984190:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f493b0 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984201:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984208:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984214:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984310:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787826270.984321:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826270.984326:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984348:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.984356:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984433:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787826270.984440:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984453:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826270.984458:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826270.984464:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984493:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984499:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826270.984534:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 1/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984547:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f493be lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f493c5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984562:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826270.984565:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984571:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826270.984575:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826270.984581:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984597:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984603:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826270.984620:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984627:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f493a2 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f493a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984640:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826270.984643:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984654:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826270.984657:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826270.984664:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984678:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984684:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826270.984690:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.984697:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826270.984699:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826270.984701:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826270.984715:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346100352:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826270.984758:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.984789:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826270.984795:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346100224:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826270.984831:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.984862:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346100352 00000100:00100000:1.0:1787826270.984868:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346100352 00000100:00100000:1.0:1787826270.984903:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346100224 00000100:00100000:1.0:1787826270.984906:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346100224 00000100:00100000:1.0:1787826270.984939:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10982 00000100:00100000:1.0:1787826270.984948:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+19:3653:x1874663346100352:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826270.984959:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.984977:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4938d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49386 expref: 19 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.985000:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826270.985003:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826270.985033:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+18:3653:x1874663346100352:12345-0@lo:103:ldlm_bl.0 Request processed in 87us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.985039:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10982 00000100:00100000:1.0:1787826270.985054:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10983 00000100:00100000:1.0:1787826270.985060:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+17:3653:x1874663346100224:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826270.985067:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.985077:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f493a9 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e3e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f493a2 expref: 17 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.985088:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:1.0:1787826270.985094:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826270.985143:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+16:3653:x1874663346100224:12345-0@lo:103:ldlm_bl.0 Request processed in 83us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.985149:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10983 00000100:00100000:1.0:1787826270.985182:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346100224:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826270.985191:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346100352:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826270.985213:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826270.985219:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346100096:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826270.985242:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826270.985268:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346100096 00000100:00100000:1.0:1787826270.985272:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346100096 00000100:00100000:1.0:1787826270.985296:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10984 00000100:00100000:1.0:1787826270.985302:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3654:x1874663346100096:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826270.985309:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826270.985320:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f493c5 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f493be expref: 16 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826270.985332:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18b:0x0].0x0 (ffff9f7952ca9000) 00010000:00010000:2.0:1787826270.985423:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826270.985922:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1787826270.985939:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f49386 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4938d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826270.985957:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826270.985963:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826270.985970:0:6529:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826270.986050:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223392:x1874663332513024:12345-192.168.203.56@tcp:36:rm.0 Request processed in 46903us (47270us total) trans 4294985322 rc 0/0 00000100:00100000:0.0:1787826270.986059:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68190 00010000:00010000:1.0:1787826270.986971:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826270.987037:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3654:x1874663346100096:12345-0@lo:103:ldlm_bl.0 Request processed in 1733us (1792us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826270.987046:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10984 00000100:00100000:0.0:1787826270.987310:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346100096:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826270.987421:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826271.089376:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826271.089545:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332513664 00000100:00100000:2.0:1787826271.089555:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332513664 00000100:00100000:2.0:1787826271.089618:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68191 00000100:00100000:2.0:1787826271.089629:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223400:x1874663332513664:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826271.089662:0:6527:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345594368 00000100:00100000:2.0:1787826271.089741:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223400:x1874663332513664:12345-192.168.203.56@tcp:41:stat.0 Request processed in 114us (374us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.089756:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68191 00000100:00100000:3.0:1787826271.136685:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826271.136890:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332513792 00000100:00100000:0.0:1787826271.136904:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332513792 00000100:00100000:0.0:1787826271.136998:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68192 00000100:00100000:0.0:1787826271.137012:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223404:x1874663332513792:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826271.137038:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826271.137066:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f493cc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e67 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.137187:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.137268:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.137284:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826271.137345:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826271.137350:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826271.137435:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.137660:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f493cc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.137685:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.137693:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826271.137696:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826271.137701:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826271.137899:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223404:x1874663332513792:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 889us (1337us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826271.137908:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68192 00000100:00100000:0.0:1787826271.145017:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826271.145766:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332513920 00000100:00100000:3.0:1787826271.145852:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332513920 00000100:00100000:3.0:1787826271.145960:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68193 00000100:00100000:3.0:1787826271.145973:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223404:x1874663332513920:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826271.146001:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826271.146029:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f493da lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e6e expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.146174:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.146322:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.146339:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.146354:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.146366:0:35093:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4939b lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49394 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.146900:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.146959:0:35093:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4939b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49394 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826271.146979:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000_UUID:35093:1874663346100480:0@lo:104: 00000100:00100000:3.0:1787826271.147053:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.147137:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826271.147342:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346100480 00000100:00100000:0.0:1787826271.147352:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346100480 00000100:00100000:0.0:1787826271.147424:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7803 00000100:00100000:0.0:1787826271.147435:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35093:x1874663346100480:12345-0@lo:104: 00010000:00010000:0.0:1787826271.147458:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.147548:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826271.147555:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35093:x1874663346100480:12345-0@lo:104: Request processed in 122us (502us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826271.147759:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826271.147767:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.147846:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826271.147853:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826271.147863:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.147899:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.147906:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826271.147940:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.147948:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f49394 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4939b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.147977:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826271.147979:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826271.147980:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826271.148012:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346100608:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826271.148052:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826271.148095:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346100608 00000100:00100000:1.0:1787826271.148101:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346100608 00010000:00010000:1.0:1787826271.148165:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4939b lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49394 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826271.148190:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21834 00000100:00100000:0.0:1787826271.148669:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7803 00000100:00100000:1.0:1787826271.148760:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+15:3654:x1874663346100608:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826271.149016:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826271.149344:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000_UUID:35093:1874663346100480:0@lo:104: 00010000:00010000:3.0:1787826271.149905:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.161503:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826271.161530:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.161605:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.161613:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826271.161707:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346100736:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826271.161724:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece600/0xe21309aa81f49355 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4935c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826271.161758:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957304 is committed 00000001:00080000:3.0:1787826271.161773:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957303 is committed 00000004:00080000:3.0:1787826271.161780:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 214 committed 00000100:00100000:0.0:1787826271.162143:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826271.162720:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826271.162742:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346100864:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1787826271.162828:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826271.162897:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346100864 00000100:00100000:0.0:1787826271.162908:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346100864 00000100:00100000:0.0:1787826271.162983:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30773 00000100:00100000:0.0:1787826271.162996:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346100864:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787826271.163132:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346100736 00000100:00100000:2.0:1787826271.163137:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346100736 00000100:00100000:2.0:1787826271.163181:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10985 00000100:00100000:2.0:1787826271.163189:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346100736:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826271.163211:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826271.163239:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4935c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49355 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.163265:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18d:0x0].0x5a734ce9 (ffff9f794b57ab00) 00010000:00010000:2.0:1787826271.163280:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00080000:3.0:1787826271.163299:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826271.163354:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346100736:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 164us (1212us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.163361:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10985 00000100:00100000:3.0:1787826271.163816:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346100736:0@lo:103:jbd2/dm-0-8.0 00002000:00080000:0.0:1787826271.163875:0:35352:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1308 count 1 00010000:00010000:0.0:1787826271.163903:0:35352:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.163937:0:35352:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.163946:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x51c:0x0].0x0 since=7s v=(0 0) 00010000:00010000:0.0:1787826271.163951:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826271.163955:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.163995:0:35352:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.164280:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4939b lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49394 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.164306:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a500) 00010000:00010000:1.0:1787826271.164312:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.164328:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.164336:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.164466:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826271.164549:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346100608:12345-0@lo:103:ldlm_bl.0 Request processed in 16351us (16495us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.164560:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21834 00000100:00100000:2.0:1787826271.164676:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346100608:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826271.164688:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.164698:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.164846:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.164856:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.164863:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.164875:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.164881:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826271.164885:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165031:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165045:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826271.165048:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826271.165053:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165060:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f493e1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165079:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.165081:0:35352:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 5 pid: 8759 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.165129:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d817100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346100992:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826271.165290:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165299:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165305:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826271.165308:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165319:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165329:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a500) 00010000:00010000:3.0:1787826271.165331:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826271.165335:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165341:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f493e8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165464:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f493da lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e6e expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165475:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826271.165478:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826271.165482:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f493da lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e6e expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.165492:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826271.165626:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223404:x1874663332513920:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 19590us (20559us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826271.165638:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68193 00000100:00100000:0.0:1787826271.165850:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ee40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826271.170966:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d817100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346100992:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826271.171001:0:35352:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.172831:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826271.172919:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332514048 00000100:00100000:1.0:1787826271.172944:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332514048 00010000:00010000:1.0:1787826271.173020:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 6 pid: 8759 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826271.173080:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21835 00000100:00100000:1.0:1787826271.173098:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332514048:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826271.173144:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826271.173151:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 6 pid: 8759 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.173167:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 6 pid: 8759 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.173197:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f49172 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800dcd expref: 6 pid: 8759 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.173232:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826271.173237:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826271.173244:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.173255:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.173280:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826271.173344:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332514048:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 248us (522us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.173352:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21835 00010000:00010000:0.0:1787826271.178080:0:35352:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.178101:0:35352:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.178143:0:35352:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.178152:0:35352:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.178189:0:35352:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826271.184968:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346100864:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21970us (22135us total) trans 4294976143 rc 0/0 00000100:00100000:0.0:1787826271.184980:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30773 00000004:00080000:2.0:1787826271.185078:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7850ad4380/2, rc 0, transno 4294976143 00000100:00100000:2.0:1787826271.185102:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346100864:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1787826271.185489:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826271.185501:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.185518:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826271.185523:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.185536:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.185639:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.185648:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000058d139e7/0xe21309aa81f493ef lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.185678:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826271.185680:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826271.185682:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826271.185966:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826271.188488:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826271.188574:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332514176 00000100:00100000:0.0:1787826271.188592:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332514176 00000100:00100000:0.0:1787826271.188681:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68194 00000100:00100000:0.0:1787826271.188695:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223404:x1874663332514176:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826271.188728:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826271.188755:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f493f6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e75 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.188850:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.188866:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.188879:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826271.188950:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826271.188954:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826271.188979:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.189008:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f493f6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.189029:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.189037:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826271.189040:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826271.189044:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826271.189140:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223404:x1874663332514176:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 448us (661us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826271.189148:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68194 00000100:00100000:0.0:1787826271.198637:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826271.198735:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332514432 00000100:00100000:2.0:1787826271.198751:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332514432 00000100:00100000:2.0:1787826271.198921:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68195 00000100:00100000:2.0:1787826271.198944:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223404:x1874663332514432:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:2.0:1787826271.199071:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346101120:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787826271.199190:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826271.199229:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.199252:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346101120 00000100:00100000:1.0:1787826271.199264:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346101120 00000100:00100000:1.0:1787826271.199324:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23243 00000100:00100000:1.0:1787826271.199334:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8d050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:11629:x1874663346101120:12345-0@lo:1000:mdt00_004.0 00000100:00080000:3.0:1787826271.199566:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826271.199898:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8d050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:11629:x1874663346101120:12345-0@lo:1000:mdt00_004.0 Request processed in 562us (705us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.199912:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23243 00000100:00100000:3.0:1787826271.199954:0:11629:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957304 gen 2 00000004:00080000:3.0:1787826271.199963:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957297 committed transno 8589957304 00000020:00080000:3.0:1787826271.199971:0:11629:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7855667c00 refcount 2 committed 0 result 0 batchid 3386 00000020:00080000:3.0:1787826271.199975:0:11629:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2e00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7855667000 00000020:00080000:3.0:1787826271.199979:0:11629:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1488 00000020:00080000:3.0:1787826271.199983:0:11629:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852be1c00 00000020:00080000:3.0:1787826271.199986:0:11629:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1170 00000004:00080000:3.0:1787826271.200040:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957298 committed transno 8589957304 00000020:00080000:3.0:1787826271.200045:0:11629:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784548d9c0 refcount 2 committed 0 result 0 batchid 3387 00000020:00080000:3.0:1787826271.200048:0:11629:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1b600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784548dcc0 00000020:00080000:3.0:1787826271.200051:0:11629:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1489 00000020:00080000:3.0:1787826271.200054:0:11629:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1b100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7847b1f200 00000020:00080000:3.0:1787826271.200057:0:11629:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1171 00000004:00080000:3.0:1787826271.200088:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957299 committed transno 8589957304 00000004:00080000:3.0:1787826271.200143:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957300 committed transno 8589957304 00000004:00080000:3.0:1787826271.200174:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957301 committed transno 8589957304 00000004:00080000:3.0:1787826271.200201:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957302 committed transno 8589957304 00000004:00080000:3.0:1787826271.200219:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957304 committed transno 8589957304 00000100:00100000:3.0:1787826271.200237:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346101120:0@lo:1000:mdt00_004.0 00000020:00080000:0.0:1787826271.200351:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3387 committed batchid 3385 00000020:00080000:0.0:1787826271.200528:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3385 00000100:00100000:3.0:1787826271.200751:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346101248:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.200939:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.201012:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.201043:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346101248 00000100:00100000:1.0:1787826271.201058:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346101248 00000100:00100000:0.0:1787826271.204917:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23244 00000100:00100000:0.0:1787826271.204958:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:11629:x1874663346101248:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787826271.205246:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:11629:x1874663346101248:12345-0@lo:1000:mdt00_004.0 Request processed in 303us (4306us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826271.205256:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23244 00000100:00100000:3.0:1787826271.210303:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346101248:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787826271.210734:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.210745:0:11629:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.210766:0:11629:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.210791:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346101376:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787826271.210855:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826271.210895:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ee40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826271.211024:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346101376 00000100:00100000:2.0:1787826271.211032:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346101376 00000100:00100000:2.0:1787826271.211092:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23245 00000100:00100000:2.0:1787826271.211102:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:11629:x1874663346101376:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787826271.211143:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826271.211159:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49404 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.211186:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.211196:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.211205:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49404 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.211232:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.211246:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 6 pid: 35098 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826271.211258:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346101504:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826271.211354:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.211370:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 6 pid: 17813 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826271.211390:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dca80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346101632:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826271.211412:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.214751:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826271.214804:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826271.215518:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332514560 00000100:00100000:3.0:1787826271.215529:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332514560 00010000:00010000:3.0:1787826271.215573:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 7 pid: 35098 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.215737:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332514688 00000100:00100000:0.0:1787826271.215743:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332514688 00000100:00100000:3.0:1787826271.215749:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21836 00010000:00010000:0.0:1787826271.215765:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 7 pid: 17813 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826271.215773:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c82c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332514560:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826271.215790:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787826271.215796:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21837 00010000:00010000:3.0:1787826271.215799:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 7 pid: 35098 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.215810:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332514688:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826271.215815:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 7 pid: 35098 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.215824:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826271.215832:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 7 pid: 17813 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.215848:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826271.215857:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49404 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.215866:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826271.215941:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c82c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332514560:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 183us (1198us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826271.215949:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21836 00010000:00010000:2.0:1787826271.217720:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f493fd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e75 expref: 6 pid: 35098 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826271.217745:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346101504:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826271.217759:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dca80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346101632:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826271.217860:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 7 pid: 17813 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.217883:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f493d3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800e67 expref: 7 pid: 17813 timeout: 8183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.217895:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826271.217901:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49404 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.217911:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49404 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.217918:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49404 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.217966:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49404 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.217980:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d814a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346101760:0@lo:105: 00000100:00100000:0.0:1787826271.218037:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826271.218081:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.218153:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826271.218166:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346101760 00000100:00100000:1.0:1787826271.218172:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346101760 00000100:00100000:1.0:1787826271.218227:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7804 00000100:00100000:1.0:1787826271.218236:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b441050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346101760:12345-0@lo:105: 00010000:00010000:1.0:1787826271.218248:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.218257:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.218265:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.218272:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.218279:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.218287:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.218299:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43c00) 00000100:00100000:1.0:1787826271.218357:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b441050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346101760:12345-0@lo:105: Request processed in 122us (320us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.218365:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7804 00000100:00100000:3.0:1787826271.218582:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d814a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346101760:0@lo:105: 00010000:00010000:3.0:1787826271.218598:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826271.218646:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332514688:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2839us (3840us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826271.218655:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21837 00010000:00010000:2.0:1787826271.221619:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49404 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.221637:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826271.221640:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826271.221644:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826271.221733:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:11629:x1874663346101376:12345-0@lo:101:mdt00_004.0 Request processed in 10631us (10877us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.221742:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23245 00000100:00100000:0.0:1787826271.221881:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346101376:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787826271.221903:0:11629:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826271.221944:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826271.221986:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.221994:0:11629:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.222002:0:11629:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.222021:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346101888:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787826271.222055:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826271.222083:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ee40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826271.222686:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346101888 00000100:00100000:2.0:1787826271.222694:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346101888 00000100:00100000:2.0:1787826271.222748:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23246 00000100:00100000:2.0:1787826271.222757:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d814e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11629:x1874663346101888:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787826271.222773:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826271.222794:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49419 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49412 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.222820:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49419 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49412 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.222833:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49419 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49412 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.222842:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe700) 00010000:00010000:2.0:1787826271.222845:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826271.222848:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826271.222965:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d814e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346101888:12345-0@lo:101:mdt00_004.0 Request processed in 208us (909us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.222972:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23246 00000100:00100000:3.0:1787826271.225962:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346101888:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787826271.225995:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.226012:0:11629:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826271.226041:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826271.226387:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dca80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102016:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787826271.226453:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826271.226493:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826271.226556:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346102016 00000100:00100000:0.0:1787826271.226563:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346102016 00000100:00100000:0.0:1787826271.226617:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23247 00000100:00100000:0.0:1787826271.226626:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102016:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787826271.226712:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102016:12345-0@lo:1000:mdt00_004.0 Request processed in 87us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826271.226719:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23247 00000100:00100000:3.0:1787826271.226793:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dca80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102016:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.226838:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102144:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.226866:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.226892:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826271.227014:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346102144 00000100:00100000:0.0:1787826271.227019:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346102144 00000100:00100000:0.0:1787826271.227052:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23248 00000100:00100000:0.0:1787826271.227059:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102144:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787826271.227165:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102144:12345-0@lo:1000:mdt00_004.0 Request processed in 106us (298us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826271.227172:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23248 00000100:00100000:3.0:1787826271.227350:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102144:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.227423:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102272:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.227445:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.227469:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826271.227569:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346102272 00000100:00100000:2.0:1787826271.227573:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346102272 00000100:00100000:2.0:1787826271.227607:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23249 00000100:00100000:2.0:1787826271.227614:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102272:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787826271.227685:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102272:12345-0@lo:1000:mdt00_004.0 Request processed in 71us (238us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826271.227693:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23249 00000100:00100000:3.0:1787826271.227790:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102272:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.227833:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102400:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.227856:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.227878:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.227940:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346102400 00000100:00100000:1.0:1787826271.227945:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346102400 00000100:00100000:1.0:1787826271.227979:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23250 00000100:00100000:1.0:1787826271.227986:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102400:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787826271.228049:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102400:12345-0@lo:1000:mdt00_004.0 Request processed in 63us (192us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.228055:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23250 00000100:00100000:3.0:1787826271.228149:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102400:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.228191:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102528:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.228226:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.228238:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.228349:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346102528 00000100:00100000:1.0:1787826271.228354:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346102528 00000100:00100000:1.0:1787826271.228386:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23251 00000100:00100000:1.0:1787826271.228392:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102528:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787826271.228450:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102528:12345-0@lo:1000:mdt00_004.0 Request processed in 58us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.228457:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23251 00000100:00100000:3.0:1787826271.228585:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102528:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.228623:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102656:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826271.228648:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.228669:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.228719:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346102656 00000100:00100000:1.0:1787826271.228723:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346102656 00000100:00100000:1.0:1787826271.228751:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23252 00000100:00100000:1.0:1787826271.228758:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102656:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787826271.228802:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:11629:x1874663346102656:12345-0@lo:1000:mdt00_004.0 Request processed in 45us (153us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.228808:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23252 00000100:00100000:3.0:1787826271.228989:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346102656:0@lo:1000:mdt00_004.0 00000040:00080000:3.0:1787826271.229764:0:11629:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1173, 2168 off1067512 00000040:00080000:3.0:1787826271.229840:0:11629:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1491, 2368 off1506808 00000004:00080000:3.0:1787826271.229847:0:11629:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7855667a80 with version 6779 00000004:00080000:0.0:1787826271.230308:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6779 rpc_version 6779 00000020:00100000:0.0:1787826271.230355:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826271.230366:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d816680 x1874663346102912/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826271.230386:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346102912:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826271.230446:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826271.230479:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ee40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.230623:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346102912 00000100:00100000:1.0:1787826271.230631:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346102912 00000100:00100000:1.0:1787826271.230679:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23253 00000100:00100000:1.0:1787826271.230689:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346102912:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826271.231847:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346102912:12345-0@lo:1000:osp_up0-1.0 Request processed in 1154us (1395us total) trans 8589957305 rc 0/0 00000100:00100000:2.0:1787826271.231857:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23253 00000100:00100000:0.0:1787826271.231974:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346102912:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787826271.232137:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.232153:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.232164:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.232182:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.232190:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.232208:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.232237:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.232245:0:11629:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.232255:0:11629:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826271.232271:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346103040:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787826271.232305:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.232503:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826271.232868:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346103040 00000100:00100000:2.0:1787826271.232876:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346103040 00000100:00100000:2.0:1787826271.232924:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23254 00000100:00100000:2.0:1787826271.232942:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:11629:x1874663346103040:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787826271.232958:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826271.232975:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49435 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4942e expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.232994:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49435 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4942e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.233005:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49435 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4942e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.233013:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826271.233016:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826271.233019:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826271.233069:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:11629:x1874663346103040:12345-0@lo:101:mdt00_004.0 Request processed in 128us (763us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.233075:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23254 00000100:00100000:0.0:1787826271.233159:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346103040:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787826271.233174:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233185:0:11629:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826271.233222:0:11629:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006d4f99df reply state 0000000037c8ae1b transno 4294985323 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233234:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233241:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:0.0:1787826271.233243:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826271.233246:0:11629:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826271.233251:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233265:0:11629:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826271.233275:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18c:0x0] 00000004:00010000:0.0:1787826271.233281:0:11629:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006d4f99df reply state 0000000037c8ae1b transno 4294985323 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233288:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233295:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:0.0:1787826271.233297:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826271.233299:0:11629:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787826271.233304:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e40:0x0]: returning size 4096 00000004:00080000:0.0:1787826271.233311:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826271.233329:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233338:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233345:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233367:0:11629:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826271.233371:0:11629:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826271.233376:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.233388:0:11629:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.233464:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223404:x1874663332514432:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 34522us (34841us total) trans 4294985323 rc 0/0 00000100:00100000:0.0:1787826271.233471:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68195 00010000:00010000:2.0:1787826271.234024:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826271.234033:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.234041:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826271.234046:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826271.234053:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.234082:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.234088:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826271.234142:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.234149:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f49404 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4940b expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.234161:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826271.234163:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826271.234164:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826271.234221:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346103168:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826271.234262:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826271.234343:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346103168 00000100:00100000:2.0:1787826271.234350:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346103168 00000100:00100000:2.0:1787826271.234389:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10986 00000100:00100000:2.0:1787826271.234397:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346103168:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826271.234408:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826271.234424:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4940b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49404 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.234438:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826271.234442:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826271.234484:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346103168:12345-0@lo:103:ldlm_bl.0 Request processed in 88us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.234491:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10986 00000100:00100000:3.0:1787826271.234614:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346103168:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826271.239567:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826271.239622:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332514816 00000100:00100000:2.0:1787826271.239633:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332514816 00000100:00100000:2.0:1787826271.239705:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68196 00000100:00100000:2.0:1787826271.239729:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223393:x1874663332514816:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826271.239752:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826271.239774:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4943c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e7c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.239840:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49443 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.239855:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49443 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.239866:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49443 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826271.239915:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826271.239919:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826271.239949:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49443 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.239973:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4943c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e7c expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.239991:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49443 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e7c expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.240000:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826271.240002:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826271.240006:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826271.240068:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223393:x1874663332514816:12345-192.168.203.56@tcp:101:bash.0 Request processed in 353us (510us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.240076:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68196 00000100:00100000:1.0:1787826271.246485:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826271.246578:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332514944 00000100:00100000:3.0:1787826271.246588:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332514944 00000100:00100000:3.0:1787826271.246651:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68197 00000100:00100000:3.0:1787826271.246662:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223393:x1874663332514944:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826271.246682:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826271.246705:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4944a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e83 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.246893:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.246907:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.246918:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.246938:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.246948:0:35095:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49419 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49412 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.246956:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.246992:0:35095:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49419 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49412 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826271.247009:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346103296:0@lo:104: 00000100:00100000:3.0:1787826271.247054:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826271.247096:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7855667840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826271.247205:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346103296 00000100:00100000:0.0:1787826271.247210:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346103296 00000100:00100000:0.0:1787826271.247249:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7805 00000100:00100000:0.0:1787826271.247257:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346103296:12345-0@lo:104: 00010000:00010000:0.0:1787826271.247269:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.247320:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826271.247326:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346103296:12345-0@lo:104: Request processed in 70us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826271.247333:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7805 00010000:00010000:2.0:1787826271.247444:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826271.247451:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.247460:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826271.247465:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826271.247473:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.247499:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.247506:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826271.247534:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.247543:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f49412 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49419 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.247562:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826271.247564:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826271.247566:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826271.247609:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346103424:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826271.247643:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826271.247725:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346103424 00000100:00100000:2.0:1787826271.247730:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346103424 00010000:00010000:2.0:1787826271.247750:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49419 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49412 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826271.247778:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21838 00000100:00100000:2.0:1787826271.247785:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346103424:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826271.247796:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826271.248816:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346103296:0@lo:104: 00010000:00010000:1.0:1787826271.248838:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826271.250243:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957305 is committed 00010000:00010000:2.0:1787826271.251946:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49419 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49412 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.251971:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe700) 00010000:00010000:2.0:1787826271.251977:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.251987:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.251995:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.252032:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826271.252149:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346103424:12345-0@lo:103:ldlm_bl.0 Request processed in 4363us (4504us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.252159:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21838 00010000:00010000:1.0:1787826271.252261:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252270:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826271.252317:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346103424:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826271.252394:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-mdtlov_UUID:35095:1874663346103552:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826271.252435:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826271.252467:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826271.252578:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346103552 00000100:00100000:2.0:1787826271.252585:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346103552 00000100:00100000:2.0:1787826271.252630:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23255 00000100:00100000:2.0:1787826271.252639:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:35095:x1874663346103552:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826271.252714:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:35095:x1874663346103552:12345-0@lo:1000:mdt00_012.0 Request processed in 76us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.252817:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-mdtlov_UUID:35095:1874663346103552:0@lo:1000:mdt00_012.0 00010000:00010000:1.0:1787826271.252861:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4945f lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252873:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4945f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252881:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4945f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826271.252901:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4945f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252913:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252924:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252942:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252948:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252954:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826271.252958:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252965:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252975:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826271.252978:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826271.252982:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.252988:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49451 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253000:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253006:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253011:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253016:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826271.253019:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253024:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253032:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe700) 00010000:00010000:1.0:1787826271.253034:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826271.253037:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253042:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49458 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253077:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4944a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e83 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253091:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4945f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e83 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.253097:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:1.0:1787826271.253100:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826271.253102:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826271.253176:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223393:x1874663332514944:12345-192.168.203.56@tcp:101:bash.0 Request processed in 6516us (6700us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.253183:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68197 00000100:00100000:2.0:1787826271.253364:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23255 00000100:00100000:1.0:1787826271.256242:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826271.256303:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332515072 00000100:00100000:3.0:1787826271.256312:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332515072 00000100:00100000:0.0:1787826271.256387:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68198 00000100:00100000:0.0:1787826271.256401:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223393:x1874663332515072:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826271.256422:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826271.256441:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49466 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e8a expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.256503:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4946d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.256520:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4946d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.256530:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4946d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826271.256632:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e40:0x0]: returning size 4096 00000004:00200000:0.0:1787826271.256638:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826271.256675:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49466 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e8a expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.256692:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4946d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e8a expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.256700:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:0.0:1787826271.256702:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826271.256705:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826271.256771:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223393:x1874663332515072:12345-192.168.203.56@tcp:101:bash.0 Request processed in 374us (538us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826271.256778:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68198 00000100:00100000:1.0:1787826271.262408:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826271.262486:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332515200 00000100:00100000:0.0:1787826271.262497:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332515200 00000100:00100000:0.0:1787826271.262557:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68199 00000100:00100000:0.0:1787826271.262568:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223393:x1874663332515200:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826271.262589:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826271.262611:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f49474 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e91 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.262675:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4947b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.262687:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4947b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.262697:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4947b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826271.262850:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18e:0x0]: returning size 4096 00000004:00200000:0.0:1787826271.262855:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826271.262890:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f49474 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e91 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.262909:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4947b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e91 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.262918:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:0.0:1787826271.262921:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826271.262924:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826271.262994:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223393:x1874663332515200:12345-192.168.203.56@tcp:101:bash.0 Request processed in 428us (594us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826271.263001:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68199 00000100:00100000:1.0:1787826271.265202:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826271.265335:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332515328 00000100:00100000:2.0:1787826271.265344:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332515328 00000100:00100000:2.0:1787826271.265401:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68200 00000100:00100000:2.0:1787826271.265411:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223393:x1874663332515328:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826271.265428:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826271.265448:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49482 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e98 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.265504:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.265518:0:35092:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49435 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4942e expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.265527:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.265559:0:35092:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49435 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4942e expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826271.265575:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346103680:0@lo:104: 00000100:00100000:2.0:1787826271.265622:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826271.265660:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826271.265856:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346103680 00000100:00100000:0.0:1787826271.265862:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346103680 00000100:00100000:0.0:1787826271.265901:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7806 00000100:00100000:0.0:1787826271.265908:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35092:x1874663346103680:12345-0@lo:104: 00010000:00010000:0.0:1787826271.265923:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.265990:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826271.265996:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35092:x1874663346103680:12345-0@lo:104: Request processed in 89us (374us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826271.266002:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7806 00000100:00100000:3.0:1787826271.266074:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346103680:0@lo:104: 00010000:00010000:3.0:1787826271.266094:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.266237:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826271.266245:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.266253:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826271.266259:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826271.266268:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.266294:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.266301:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826271.266328:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 1/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.266336:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4942e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49435 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826271.266357:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826271.266359:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826271.266361:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826271.266448:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826271.266461:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346103808:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826271.266498:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826271.266572:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346103808 00000100:00100000:2.0:1787826271.266577:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346103808 00010000:00010000:2.0:1787826271.266597:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49435 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4942e expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826271.266633:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21839 00000100:00100000:2.0:1787826271.266641:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346103808:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826271.266652:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826271.267386:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49435 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4942e expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.267402:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826271.267407:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.267415:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.267421:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.267488:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826271.267545:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346103808:12345-0@lo:103:ldlm_bl.0 Request processed in 904us (1045us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.267553:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21839 00000100:00100000:1.0:1787826271.269154:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346103808:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826271.273506:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.273525:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826271.273674:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18c:0x0]: returning size 4096 00000004:00200000:2.0:1787826271.273680:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826271.273805:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49482 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e98 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.273829:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e98 expref: 9 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.273844:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826271.273847:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826271.273851:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826271.273973:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223393:x1874663332515328:12345-192.168.203.56@tcp:101:bash.0 Request processed in 8562us (8777us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.273981:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68200 00000100:00100000:0.0:1787826271.292185:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826271.292292:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332515584 00000100:00100000:3.0:1787826271.292303:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332515584 00000100:00100000:3.0:1787826271.292365:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68201 00000100:00100000:3.0:1787826271.292378:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d814a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223393:x1874663332515584:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826271.292402:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826271.292405:0:35098:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826271.292430:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49489 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e98 expref: 9 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.292449:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:3.0:1787826271.292462:0:35098:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826271.292478:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49490 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800e9f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.292604:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.292617:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.292626:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.292637:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.292644:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.292652:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826271.293095:0:35098:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x51c:0x0], index 0: 0 00010000:00010000:3.0:1787826271.293313:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f494a5 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293325:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f494a5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293333:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f494a5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293364:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293371:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293378:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293384:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293390:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826271.293395:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293402:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293411:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f79460a2600) 00010000:00010000:3.0:1787826271.293414:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826271.293418:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293424:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49497 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826271.293435:0:35098:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009958f294 reply state 000000003a4a7249 transno 8589957306 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293444:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293450:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826271.293452:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826271.293454:0:35098:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826271.293481:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49490 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e9f expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293496:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f494a5 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e9f expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.293502:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f79460a2100) 00010000:00010000:3.0:1787826271.293505:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826271.293506:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826271.293617:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d814a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223393:x1874663332515584:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1242us (1441us total) trans 8589957306 rc 0/0 00000100:00100000:3.0:1787826271.293625:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68201 00000100:00100000:1.0:1787826271.305036:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826271.305136:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332515712 00000100:00100000:2.0:1787826271.305146:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332515712 00000100:00100000:2.0:1787826271.305224:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30774 00000100:00100000:2.0:1787826271.305235:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796eda4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:223393:x1874663332515712:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826271.305253:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826271.305276:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ea6 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826271.305301:0:17826:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x51c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826271.305322:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=8083s v=(0 0) 00010000:00010000:2.0:1787826271.305326:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826271.305336:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.305351:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.305363:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826271.305424:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796eda4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223393:x1874663332515712:12345-192.168.203.56@tcp:101:bash.0 Request processed in 191us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.305432:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30774 00000100:00080000:3.0:1787826271.305630:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826271.312560:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826271.312660:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332515968 00000100:00100000:2.0:1787826271.312671:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332515968 00000100:00100000:0.0:1787826271.312804:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9296 00000100:00100000:0.0:1787826271.312818:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223393:x1874663332515968:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826271.313156:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223393:x1874663332515968:12345-192.168.203.56@tcp:35:bash.0 Request processed in 341us (605us total) trans 8589957307 rc 0/0 00000100:00100000:0.0:1787826271.313164:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9296 00000100:00100000:1.0:1787826271.321534:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826271.321637:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332516224 00000100:00100000:0.0:1787826271.321646:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332516224 00000100:00100000:0.0:1787826271.321714:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68202 00000100:00100000:0.0:1787826271.321840:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223393:x1874663332516224:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826271.321861:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826271.321865:0:8752:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826271.321894:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4947b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e91 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.321913:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:0.0:1787826271.321918:0:8752:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826271.321939:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f494ba lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ead expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.322060:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.322074:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.322084:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.322101:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.322165:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.322173:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826271.322299:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d817100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346103936:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787826271.322355:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826271.322383:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ed80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826271.322498:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346103936 00000100:00100000:1.0:1787826271.322504:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346103936 00000100:00100000:1.0:1787826271.322544:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23256 00000100:00100000:1.0:1787826271.322552:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:8752:x1874663346103936:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826271.322627:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:8752:x1874663346103936:12345-0@lo:1000:mdt00_003.0 Request processed in 75us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826271.322635:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23256 00000100:00100000:0.0:1787826271.322807:0:8752:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957305 gen 2 00000004:00080000:0.0:1787826271.322813:0:8752:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957305 committed transno 8589957305 00000100:00100000:0.0:1787826271.322843:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d817100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346103936:0@lo:1000:mdt00_003.0 00000004:00080000:0.0:1787826271.323143:0:8752:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4c7:0x0], index 1: 0 00010000:00010000:0.0:1787826271.323493:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f494cf lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323506:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f494cf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323516:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f494cf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323551:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323559:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323565:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323571:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323577:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826271.323582:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323588:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323599:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:0.0:1787826271.323602:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826271.323606:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323613:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f494c1 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826271.323626:0:8752:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b20713aa reply state 000000005445510a transno 4294985324 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323637:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323643:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x4ec94fd6 (ffff9f794b57a000) 00010000:00010000:0.0:1787826271.323645:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826271.323648:0:8752:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826271.323679:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f494ba lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ead expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323694:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f494cf lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ead expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826271.323701:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:0.0:1787826271.323704:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826271.323706:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826271.323890:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223393:x1874663332516224:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2054us (2365us total) trans 4294985324 rc 0/0 00000100:00100000:0.0:1787826271.326550:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68202 00000100:00100000:1.0:1787826271.337907:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826271.338034:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332516352 00000100:00100000:3.0:1787826271.338044:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332516352 00000100:00100000:3.0:1787826271.338316:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30775 00000100:00100000:3.0:1787826271.338329:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:223393:x1874663332516352:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826271.338347:0:35097:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826271.338371:0:35097:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800eb4 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826271.338397:0:35097:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4c7:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1787826271.338419:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c7:0x0].0x0 since=8083s v=(0 0) 00010000:00010000:3.0:1787826271.338424:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826271.338435:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.338451:0:35097:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.338464:0:35097:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826271.338529:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223393:x1874663332516352:12345-192.168.203.56@tcp:101:bash.0 Request processed in 204us (633us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826271.338537:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30775 00000100:00100000:2.0:1787826271.347996:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826271.348078:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332516608 00000100:00100000:3.0:1787826271.348088:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332516608 00000100:00100000:3.0:1787826271.348165:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9297 00000100:00100000:3.0:1787826271.348176:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223393:x1874663332516608:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826271.348477:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:223393:x1874663332516608:12345-192.168.203.56@tcp:35:bash.0 Request processed in 302us (490us total) trans 4294985325 rc 0/0 00000100:00100000:3.0:1787826271.348486:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9297 00000100:00100000:1.0:1787826271.380483:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826271.380575:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332516736 00000100:00100000:2.0:1787826271.380585:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332516736 00000100:00100000:2.0:1787826271.380652:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10987 00000100:00100000:2.0:1787826271.380664:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:15673:x1874663332516736:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826271.380679:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826271.380706:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49443 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e7c expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.380739:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826271.380751:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f494a5 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e9f expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.380761:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f79460a2100) 00010000:00010000:2.0:1787826271.380781:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4945f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e83 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826271.380792:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:2.0:1787826271.380798:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826271.380860:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:15673:x1874663332516736:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 199us (386us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826271.380868:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10987 00000100:00100000:2.0:1787826271.382537:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826271.382582:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826271.382658:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332516864 00000100:00100000:3.0:1787826271.382666:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332516864 00000100:00100000:3.0:1787826271.382733:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332516992 00000100:00100000:3.0:1787826271.382736:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332516992 00000100:00100000:3.0:1787826271.382767:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10988 00000100:00100000:3.0:1787826271.382778:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:15672:x1874663332516864:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826271.382798:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826271.382818:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f492c2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e3f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e2f expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.382835:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3f:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:3.0:1787826271.382845:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826271.382890:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:15672:x1874663332516864:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 114us (362us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826271.382897:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10988 00000100:00100000:3.0:1787826271.382915:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10989 00000100:00100000:3.0:1787826271.382922:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c80050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:34879:x1874663332516992:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826271.382941:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826271.382952:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f494cf lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ead expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.382965:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:3.0:1787826271.382976:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4946d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e8a expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.382985:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:3.0:1787826271.382993:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4921a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e3e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800e0c expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826271.383001:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e3e:0x0].0x0 (ffff9f7961d81e00) 00010000:00010000:3.0:1787826271.383010:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826271.383027:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c80050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:34879:x1874663332516992:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 106us (445us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826271.383034:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10989 00010000:00010000:0.0:1787826271.404380:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826272.440699:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826272.689656:0:164686:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:2.0:1787826272.689704:0:164686:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:0.0:1787826273.452341:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826273.503841:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826273.503892:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332517120 00000100:00100000:0.0:1787826273.503905:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332517120 00000100:00100000:0.0:1787826273.503986:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2206 00000100:00100000:0.0:1787826273.503999:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847251450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2393:x1874663332517120:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826273.504094:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847251450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2393:x1874663332517120:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 98us (268us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826273.504200:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2206 00000100:00100000:1.0:1787826273.530342:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826273.530397:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826273.530425:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826273.530450:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826273.530472:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826273.530492:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826273.530518:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826273.530553:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332517248 00000100:00100000:1.0:1787826273.530565:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332517248 00000100:00100000:1.0:1787826273.530637:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332517760 00000100:00100000:1.0:1787826273.530641:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332517760 00000100:00100000:1.0:1787826273.530661:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332517376 00000100:00100000:1.0:1787826273.530664:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332517376 00000100:00100000:1.0:1787826273.530698:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332517632 00000100:00100000:1.0:1787826273.530702:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332517632 00000100:00100000:1.0:1787826273.530726:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14221 00000100:00100000:1.0:1787826273.530737:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332517248:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826273.530824:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332517248:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 88us (497us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826273.530833:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14221 00000100:00100000:1.0:1787826273.530853:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14222 00000100:00100000:1.0:1787826273.530860:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332517760:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826273.530892:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332517760:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 33us (467us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826273.530899:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14222 00000100:00100000:1.0:1787826273.530925:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14223 00000100:00100000:1.0:1787826273.530932:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332517376:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826273.530957:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332517376:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (485us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826273.530964:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14223 00000100:00100000:1.0:1787826273.530974:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14224 00000100:00100000:1.0:1787826273.530981:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332517632:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826273.531011:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332517632:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (517us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826273.531017:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14224 00000100:00100000:1.0:1787826273.531037:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332517504 00000100:00100000:1.0:1787826273.531041:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332517504 00000100:00100000:1.0:1787826273.531072:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332518016 00000100:00100000:1.0:1787826273.531075:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332518016 00000100:00100000:1.0:1787826273.531093:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332517888 00000100:00100000:1.0:1787826273.531096:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332517888 00000100:00100000:1.0:1787826273.531143:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8756 00000100:00100000:1.0:1787826273.531151:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332517504:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826273.531180:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332517504:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 31us (782us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826273.531188:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8756 00000100:00100000:1.0:1787826273.531198:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8757 00000100:00100000:1.0:1787826273.531205:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332518016:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826273.531228:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332518016:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (778us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826273.531234:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8757 00000100:00100000:1.0:1787826273.531245:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8758 00000100:00100000:1.0:1787826273.531251:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332517888:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826273.531273:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332517888:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (754us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826273.531279:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8758 00000100:00080000:1.0:1787826273.531308:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826273.531312:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826273.531316:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826273.531318:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826274.284621:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d817100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346104064:0@lo:400:kworker.0 00000100:00100000:1.0:1787826274.284757:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826274.285158:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346104448:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.285203:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826274.285227:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346104960:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.285250:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826274.285267:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346105472:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.285293:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826274.285317:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1787826274.285323:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3655:1874663346104320:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.285348:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826274.285368:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346104832:0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.285521:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346104448 00000100:00100000:3.0:1787826274.285530:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346104448 00000100:00100000:3.0:1787826274.285598:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346104960 00000100:00100000:3.0:1787826274.285601:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346104960 00000100:00100000:3.0:1787826274.285619:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346104320 00000100:00100000:3.0:1787826274.285621:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346104320 00000100:00100000:3.0:1787826274.285645:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14225 00000100:00100000:3.0:1787826274.285656:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346104448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.285722:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346104448:12345-0@lo:400:kworker.0 Request processed in 67us (517us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826274.285731:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14225 00000100:00100000:3.0:1787826274.285754:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14226 00000100:00100000:3.0:1787826274.285761:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346104960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.285791:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346104960:12345-0@lo:400:kworker.0 Request processed in 31us (540us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826274.285797:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14226 00000100:00100000:3.0:1787826274.285810:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14227 00000100:00100000:3.0:1787826274.285823:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3655:x1874663346104320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.285849:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3655:x1874663346104320:12345-0@lo:400:kworker.0 Request processed in 34us (500us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826274.285855:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14227 00000100:00080000:3.0:1787826274.285876:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826274.285880:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826274.285902:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346104192:0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.285947:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826274.285967:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346104704:0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.285991:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826274.286012:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346105216:0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.286035:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826274.286056:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1787826274.286062:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346104576:0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.286088:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826274.286141:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346105088:0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.286165:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826274.286184:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465cbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346105600:0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.286205:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826274.286237:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346105472 00000100:00100000:3.0:1787826274.286243:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346105472 00000100:00100000:1.0:1787826274.286941:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346104576 00000100:00100000:1.0:1787826274.286952:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346104576 00000100:00100000:1.0:1787826274.286995:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346105600 00000100:00100000:1.0:1787826274.286998:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346105600 00000100:00100000:1.0:1787826274.287020:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14228 00000100:00100000:1.0:1787826274.287030:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346104576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826274.287128:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346104576:12345-0@lo:400:kworker.0 Request processed in 76us (1015us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826274.287135:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14228 00000100:00100000:1.0:1787826274.287152:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14229 00000100:00100000:1.0:1787826274.287159:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346105600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826274.287191:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346105600:12345-0@lo:400:kworker.0 Request processed in 33us (985us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826274.287197:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14229 00000100:00100000:1.0:1787826274.287219:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346104704 00000100:00100000:1.0:1787826274.287224:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346104704 00000100:00100000:3.0:1787826274.287293:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346105088 00000100:00100000:3.0:1787826274.287297:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346105088 00000100:00100000:0.0:1787826274.287396:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826274.287428:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465ca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346105344:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.287458:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826274.287503:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346104448:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.287519:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346104960:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.287554:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3655:1874663346104320:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.287579:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346104832 00000100:00100000:0.0:1787826274.287582:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346104832 00000100:00100000:0.0:1787826274.287617:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14230 00000100:00100000:0.0:1787826274.287624:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346104832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.287665:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346104832:12345-0@lo:400:kworker.0 Request processed in 41us (268us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826274.287674:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14230 00000100:00100000:0.0:1787826274.287697:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346104192 00000100:00100000:0.0:1787826274.287702:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346104192 00000100:00100000:0.0:1787826274.287730:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346105216 00000100:00100000:0.0:1787826274.287734:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346105216 00000100:00100000:0.0:1787826274.287754:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23257 00000100:00100000:0.0:1787826274.287761:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346104192:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.287796:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346104192:12345-0@lo:400:kworker.0 Request processed in 36us (1849us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826274.287802:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23257 00000100:00100000:0.0:1787826274.287814:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23258 00000100:00100000:0.0:1787826274.287821:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346105216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.287849:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346105216:12345-0@lo:400:kworker.0 Request processed in 29us (1813us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826274.287854:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23258 00000100:00100000:0.0:1787826274.287880:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346104832:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.288055:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346105344 00000100:00100000:0.0:1787826274.288059:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346105344 00000100:00100000:2.0:1787826274.288145:0:30904:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346104064 00000100:00100000:2.0:1787826274.288150:0:30904:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346104064 00000100:00100000:2.0:1787826274.288186:0:30904:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2207 00000100:00100000:2.0:1787826274.288193:0:30904:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847802050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346104064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826274.288243:0:30904:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847802050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346104064:12345-0@lo:400:kworker.0 Request processed in 50us (3486us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826274.288250:0:30904:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2207 00000100:00100000:2.0:1787826274.288311:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d817100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346104064:0@lo:400:kworker.0 00000100:00100000:2.0:1787826274.288362:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8759 00000100:00100000:2.0:1787826274.288371:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346105472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826274.288413:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346105472:12345-0@lo:400:kworker.0 Request processed in 44us (3118us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826274.288419:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8759 00000100:00100000:1.0:1787826274.290561:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8760 00000100:00100000:1.0:1787826274.290593:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346104704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826274.290708:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346104704:12345-0@lo:400:kworker.0 Request processed in 133us (4715us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826274.290716:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8760 00000100:00100000:1.0:1787826274.290819:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346105472:0@lo:400:kworker.0 00000100:00080000:1.0:1787826274.290863:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787826274.290870:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826274.290873:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826274.290876:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826274.290899:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346104576:0@lo:400:kworker.0 00000100:00100000:1.0:1787826274.290926:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465cbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346105600:0@lo:400:kworker.0 00000100:00100000:1.0:1787826274.290940:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346104192:0@lo:400:kworker.0 00000100:00100000:1.0:1787826274.290952:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346104704:0@lo:400:kworker.0 00000100:00100000:1.0:1787826274.290969:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346105216:0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.291098:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8761 00000100:00100000:3.0:1787826274.291128:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346105088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826274.291181:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346105088:12345-0@lo:400:kworker.0 Request processed in 76us (5015us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826274.291188:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8761 00000100:00100000:3.0:1787826274.291216:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346105088:0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.291316:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8762 00000100:00100000:0.0:1787826274.291328:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346105344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826274.291503:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346105344:12345-0@lo:400:kworker.0 Request processed in 50us (3914us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826274.291512:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8762 00000100:00100000:0.0:1787826274.291549:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465ca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346105344:0@lo:400:kworker.0 00010000:00010000:1.0:1787826274.477447:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826274.796412:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826274.796444:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346105728:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826274.796538:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826274.796772:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346105728 00000100:00100000:3.0:1787826274.796784:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346105728 00000100:00100000:3.0:1787826274.796856:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6195 00000100:00100000:3.0:1787826274.796869:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346105728:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826274.796917:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826274.796977:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346105728:12345-0@lo:13:osp-pre-1-1.0 Request processed in 109us (437us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826274.796984:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6195 00000100:00100000:0.0:1787826274.797326:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346105728:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826275.306273:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826275.308859:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346105856:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826275.308963:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826275.309036:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346105856 00000100:00100000:3.0:1787826275.309047:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346105856 00000100:00100000:3.0:1787826275.309136:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6196 00000100:00100000:3.0:1787826275.309151:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346105856:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826275.309192:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826275.309212:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346106112:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826275.309456:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332518144 00000100:00100000:0.0:1787826275.309462:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332518144 00000100:00100000:2.0:1787826275.316362:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826275.316411:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346106112 00000100:00100000:1.0:1787826275.316424:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346106112 00000100:00100000:2.0:1787826275.316437:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826275.316452:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346106240:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826275.316493:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23259 00000100:00100000:1.0:1787826275.316506:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346106112:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826275.316557:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443442688 avail 1297203200 00000100:00100000:1.0:1787826275.316629:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346106112:12345-0@lo:41:osp-pre-1-0.0 Request processed in 124us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.316637:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23259 00000100:00100000:2.0:1787826275.316676:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826275.316734:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346106112:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826275.316770:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346105984:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826275.316796:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826275.316813:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826275.316893:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346106368:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826275.316932:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826275.316963:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346106368 00000100:00100000:2.0:1787826275.316968:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346106368 00000100:00100000:2.0:1787826275.316998:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23260 00000100:00100000:2.0:1787826275.317004:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346106368:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826275.317027:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442828288 avail 1296588800 00000100:00100000:2.0:1787826275.317066:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346106368:12345-0@lo:41:osp-pre-0-1.0 Request processed in 61us (133us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.317072:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23260 00000100:00100000:2.0:1787826275.317129:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346106368:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826275.317193:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68203 00000100:00100000:2.0:1787826275.317200:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223574:x1874663332518144:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826275.317212:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826275.317235:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f494e4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ebb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.317380:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.317397:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.317412:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826275.317684:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826275.317690:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826275.317716:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.317750:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f494e4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.317771:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.317781:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826275.317785:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826275.317789:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826275.318043:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223574:x1874663332518144:12345-192.168.203.56@tcp:101:cat.0 Request processed in 841us (11784us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.318050:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68203 00000100:00100000:1.0:1787826275.318232:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346106240 00000100:00100000:1.0:1787826275.318242:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346106240 00000100:00100000:1.0:1787826275.318274:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346105984 00000100:00100000:1.0:1787826275.318277:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346105984 00000100:00100000:2.0:1787826275.318293:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6197 00000100:00100000:2.0:1787826275.318305:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346106240:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826275.318325:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826275.318385:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346106240:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (1710us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.321731:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346105856:12345-0@lo:13:osp-pre-0-1.0 Request processed in 12567us (12751us total) trans 0 rc 0/0 00000100:00080000:1.0:1787826275.324630:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826275.324942:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6197 00000100:00100000:3.0:1787826275.324996:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6196 00000100:00100000:3.0:1787826275.325030:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6198 00000100:00100000:3.0:1787826275.325054:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346105984:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787826275.325096:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826275.325235:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346105984:12345-0@lo:13:osp-pre-1-0.0 Request processed in 181us (8425us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.325244:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346105856:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826275.325245:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6198 00000100:00100000:1.0:1787826275.325306:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346106240:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826275.329895:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346105984:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826275.333320:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826275.333473:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332518272 00000100:00100000:3.0:1787826275.333487:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332518272 00000100:00100000:3.0:1787826275.333571:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68204 00000100:00100000:3.0:1787826275.333586:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223574:x1874663332518272:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826275.333614:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826275.333642:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f494f2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ec2 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333726:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333743:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333756:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333774:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333781:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333787:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333879:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49507 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333889:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49507 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333911:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49507 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826275.333939:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49507 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333961:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333968:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333974:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333980:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333986:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826275.333991:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.333998:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334012:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826275.334015:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826275.334020:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334026:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f494f9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334041:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334047:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334053:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334057:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826275.334059:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334066:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334072:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961d81900) 00010000:00010000:3.0:1787826275.334075:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826275.334077:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334082:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49500 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334148:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f494f2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ec2 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334163:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49507 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ec2 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.334184:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:3.0:1787826275.334187:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826275.334190:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:1.0:1787826275.334959:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223574:x1874663332518272:12345-192.168.203.56@tcp:101:cat.0 Request processed in 1371us (1648us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.334971:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68204 00000100:00100000:1.0:1787826275.343674:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826275.344027:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332518400 00000100:00100000:3.0:1787826275.344040:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332518400 00000100:00100000:3.0:1787826275.344164:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68205 00000100:00100000:3.0:1787826275.344187:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223574:x1874663332518400:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826275.344214:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826275.344240:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4950e lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ec9 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.344322:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f49515 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.344338:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f49515 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.344354:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f49515 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826275.344488:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e40:0x0]: returning size 4096 00000004:00200000:3.0:1787826275.344494:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826275.344541:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4950e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ec9 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.344564:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f49515 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ec9 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.344574:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:3.0:1787826275.344577:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826275.344582:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826275.344669:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223574:x1874663332518400:12345-192.168.203.56@tcp:101:cat.0 Request processed in 487us (1011us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.344677:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68205 00000100:00080000:1.0:1787826275.345279:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826275.350579:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826275.359645:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332518528 00000100:00100000:0.0:1787826275.359667:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332518528 00000100:00100000:0.0:1787826275.359756:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68206 00000100:00100000:0.0:1787826275.359773:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223574:x1874663332518528:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826275.359802:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826275.359832:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4951c lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ed0 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.359919:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.359937:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.359952:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826275.360012:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18e:0x0]: returning size 4096 00000004:00200000:0.0:1787826275.360018:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826275.360063:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4951c lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.360084:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.360101:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:0.0:1787826275.360131:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826275.360136:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826275.360245:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223574:x1874663332518528:12345-192.168.203.56@tcp:101:cat.0 Request processed in 475us (9681us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826275.360254:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68206 00000100:00100000:2.0:1787826275.365015:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826275.365189:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332518656 00000100:00100000:0.0:1787826275.365202:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332518656 00000100:00100000:0.0:1787826275.365285:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68207 00000100:00100000:0.0:1787826275.365299:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223574:x1874663332518656:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826275.365325:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826275.365354:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4952a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ed7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.365442:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.365460:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.365475:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826275.365536:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18c:0x0]: returning size 4096 00000004:00200000:0.0:1787826275.365541:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826275.365591:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4952a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.365619:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.365629:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:0.0:1787826275.365632:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826275.365637:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826275.365720:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223574:x1874663332518656:12345-192.168.203.56@tcp:101:cat.0 Request processed in 424us (718us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826275.365727:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68207 00000100:00100000:0.0:1787826275.376667:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826275.377563:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332518912 00000100:00100000:2.0:1787826275.377579:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332518912 00000100:00100000:2.0:1787826275.377662:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68208 00000100:00100000:2.0:1787826275.377677:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223574:x1874663332518912:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826275.377711:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826275.377741:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49538 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ede expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.377858:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.377875:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.377889:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.377914:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.377921:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.377928:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:2.0:1787826275.378018:0:6528:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00010000:00010000:0.0:1787826275.501309:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826275.831813:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826275.831971:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332519040 00000100:00100000:1.0:1787826275.831984:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332519040 00000100:00100000:1.0:1787826275.832079:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68209 00000100:00100000:1.0:1787826275.832094:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223576:x1874663332519040:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826275.832400:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826275.832429:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4954d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ee5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826275.832618:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787826275.833085:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.833132:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.833149:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826275.833625:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826275.833632:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826275.833665:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.833704:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4954d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.833730:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.833740:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826275.833743:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826275.833748:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826275.833840:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223576:x1874663332519040:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1748us (2041us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.833854:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68209 00000100:00100000:0.0:1787826275.842914:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826275.843799:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332519168 00000100:00100000:2.0:1787826275.843821:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332519168 00000100:00100000:2.0:1787826275.843931:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68210 00000100:00100000:2.0:1787826275.843947:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223576:x1874663332519168:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826275.843977:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826275.844007:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4955b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800eec expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844097:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844644:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844660:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844684:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844692:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844698:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844800:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844809:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844817:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826275.844849:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844877:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844885:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844892:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844910:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844917:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826275.844922:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844930:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844943:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826275.844946:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826275.844951:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844957:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49562 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844975:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844981:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844987:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.844993:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826275.844995:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.845001:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.845008:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe800) 00010000:00010000:2.0:1787826275.845010:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826275.845012:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.845018:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49569 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.845060:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4955b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 12 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.845074:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.845083:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:2.0:1787826275.845085:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826275.845088:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826275.848407:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223576:x1874663332519168:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 4444us (5494us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.848428:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68210 00000100:00100000:0.0:1787826275.879523:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826275.879741:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332519296 00000100:00100000:1.0:1787826275.879750:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332519296 00000100:00100000:1.0:1787826275.879876:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68211 00000100:00100000:1.0:1787826275.879887:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223576:x1874663332519296:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826275.879923:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826275.879943:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f49577 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ef3 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.880001:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4957e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.880030:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4957e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.880043:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4957e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826275.880182:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000004:00200000:1.0:1787826275.880479:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e40:0x0]: returning size 4096 00000004:00200000:1.0:1787826275.880486:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826275.880526:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f49577 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ef3 expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.880545:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4957e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ef3 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.880566:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:1.0:1787826275.880568:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826275.880573:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826275.880634:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223576:x1874663332519296:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 749us (1121us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.880641:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68211 00000100:00100000:3.0:1787826275.891732:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826275.892505:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332519424 00000100:00100000:1.0:1787826275.892517:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332519424 00000100:00100000:0.0:1787826275.892827:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68212 00000100:00100000:0.0:1787826275.892840:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223576:x1874663332519424:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826275.892861:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826275.892884:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49585 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800efa expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.892974:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.892989:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893000:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893030:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893039:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893046:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893542:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4959a lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893622:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4959a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893631:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4959a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826275.893673:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e41:0x0]: returning size 6 00000004:00200000:0.0:1787826275.893677:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826275.893697:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4959a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893709:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4959a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893718:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893724:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893735:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893740:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893746:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.893750:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893756:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893768:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:0.0:1787826275.893771:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.893776:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893782:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4958c lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893795:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893800:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893806:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893811:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.893814:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893819:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893826:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x4ec94fd6 (ffff9f794b57a000) 00010000:00010000:0.0:1787826275.893828:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.893830:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893835:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f49593 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893863:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49585 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800efa expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893876:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4959a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800efa expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.893884:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:0.0:1787826275.893886:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826275.893889:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:0.0:1787826275.893976:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223576:x1874663332519424:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1137us (2253us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826275.893984:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68212 00000100:00100000:3.0:1787826275.899313:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826275.899436:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332519552 00000100:00100000:1.0:1787826275.899446:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332519552 00000100:00100000:1.0:1787826275.899515:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68213 00000100:00100000:1.0:1787826275.899527:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223576:x1874663332519552:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826275.899791:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826275.899816:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f495a1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f01 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900596:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900613:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900626:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900638:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900644:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900650:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900746:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f495b6 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900753:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f495b6 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900760:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f495b6 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826275.900797:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db4:0x0]: returning size 6 00000004:00200000:0.0:1787826275.900801:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826275.900820:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f495b6 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900830:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f495b6 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900838:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900848:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900853:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900859:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900864:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.900868:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900874:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900886:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:0.0:1787826275.900889:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.900892:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900909:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f495a8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900922:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900927:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900932:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900937:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.900939:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900945:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900951:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f79460a2e00) 00010000:00010000:0.0:1787826275.900953:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.900955:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900960:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f495af lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.900988:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f495a1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f01 expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.901002:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f495b6 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f01 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.901009:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f794b57ac00) 00010000:00010000:0.0:1787826275.901017:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826275.901019:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826275.901088:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223576:x1874663332519552:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1562us (1797us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826275.901095:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68213 00000100:00100000:1.0:1787826275.908144:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826275.910022:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826275.910140:0:125380:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332519936 00000100:00100000:0.0:1787826275.910150:0:125380:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332519936 00000100:00100000:0.0:1787826275.910214:0:125380:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1223 00000100:00100000:0.0:1787826275.910224:0:125380:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223576:x1874663332519936:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:0.0:1787826275.910275:0:125380:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826275.910314:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f495b6 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f01 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.910343:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f794b57ac00) 00010000:00010000:0.0:1787826275.910354:0:125380:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826275.910453:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346106496:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.910506:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.910545:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.911137:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346106496 00000100:00100000:3.0:1787826275.911145:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346106496 00000100:00100000:3.0:1787826275.911194:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23261 00000100:00100000:3.0:1787826275.911203:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346106496:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.911298:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346106496:12345-0@lo:1000:mdt_io00_003.0 Request processed in 96us (790us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.911305:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23261 00000100:00100000:0.0:1787826275.912881:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346106496:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.912997:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346106624:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.913076:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.913130:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.914179:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346106624 00000100:00100000:3.0:1787826275.914189:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346106624 00000100:00100000:3.0:1787826275.914242:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23262 00000100:00100000:3.0:1787826275.914251:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346106624:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.914745:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346106624:12345-0@lo:1000:mdt_io00_003.0 Request processed in 492us (1667us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.914753:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23262 00000100:00100000:0.0:1787826275.915547:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346106624:0@lo:1000:mdt_io00_003.0 00000004:00080000:0.0:1787826275.915599:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18e:0x0] 00000100:00100000:0.0:1787826275.915649:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346106752:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.915698:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.915728:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.915843:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346106752 00000100:00100000:3.0:1787826275.915850:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346106752 00000100:00100000:3.0:1787826275.915901:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23263 00000100:00100000:3.0:1787826275.915911:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346106752:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.916001:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346106752:12345-0@lo:1000:mdt_io00_003.0 Request processed in 91us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.916008:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23263 00000100:00100000:0.0:1787826275.916692:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346106752:0@lo:1000:mdt_io00_003.0 00000004:00080000:0.0:1787826275.916723:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e40:0x0] 00000100:00100000:0.0:1787826275.916758:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346106880:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.916792:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.916820:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826275.916887:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332519680 00000100:00100000:2.0:1787826275.916904:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332519680 00000100:00100000:2.0:1787826275.916956:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10990 00000100:00100000:2.0:1787826275.916965:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:2394:x1874663332519680:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:2.0:1787826275.916977:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826275.917002:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4959a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800efa expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.917037:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:2.0:1787826275.917049:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826275.917120:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2394:x1874663332519680:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 139us (8967us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.917128:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10990 00000100:00100000:3.0:1787826275.919762:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346106880 00000100:00100000:3.0:1787826275.919777:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346106880 00000100:00100000:3.0:1787826275.919835:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23264 00000100:00100000:3.0:1787826275.919845:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346106880:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.919987:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346106880:12345-0@lo:1000:mdt_io00_003.0 Request processed in 144us (3194us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.919994:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23264 00000100:00100000:0.0:1787826275.920100:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346106880:0@lo:1000:mdt_io00_003.0 00000004:00080000:0.0:1787826275.920196:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18e:0x0] 00000100:00100000:0.0:1787826275.920237:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346107008:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.920284:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.920311:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.920778:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346107008 00000100:00100000:3.0:1787826275.920785:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346107008 00000100:00100000:3.0:1787826275.920829:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23265 00000100:00100000:3.0:1787826275.920843:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346107008:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.920929:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346107008:12345-0@lo:1000:mdt_io00_003.0 Request processed in 92us (644us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.920936:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23265 00000100:00100000:0.0:1787826275.921595:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346107008:0@lo:1000:mdt_io00_003.0 00000004:00020000:0.0:1787826275.921640:0:125380:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:0.0:1787826275.921695:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.921705:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.921713:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826275.921728:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346107136:0@lo:101:mdt_io00_003.0 00000100:00100000:0.0:1787826275.921764:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.921792:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153d80 going to sleep for 11 seconds 00000004:00020000:2.0:1787826275.921902:0:6528:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4456 00010000:00010000:2.0:1787826275.921964:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.921978:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.921988:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826275.921999:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db4:0x0]: returning size 6 00010000:00010000:2.0:1787826275.922135:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922147:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922154:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922160:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922174:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826275.922179:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922186:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922198:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:2.0:1787826275.922201:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826275.922206:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922213:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4953f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922227:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922233:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922240:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922245:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826275.922248:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922253:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922620:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f79460a2e00) 00010000:00010000:2.0:1787826275.922623:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826275.922626:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922633:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49546 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922685:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49538 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922699:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.922707:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f784abfe800) 00010000:00010000:2.0:1787826275.922709:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826275.922711:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826275.922772:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223574:x1874663332518912:12345-192.168.203.56@tcp:101:cat.0 Request processed in 545099us (546121us total) trans 8589957308 rc 0/0 00000100:00100000:2.0:1787826275.922780:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68208 00000100:00100000:3.0:1787826275.923305:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346107136 00000100:00100000:3.0:1787826275.923314:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346107136 00000100:00100000:3.0:1787826275.923359:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23266 00000100:00100000:3.0:1787826275.923367:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346107136:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:3.0:1787826275.923380:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826275.923395:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f495bd expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.923691:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 2/0,1 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.923704:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.923713:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 5/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f495bd expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.923752:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.923765:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 8 pid: 8752 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826275.923778:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346107264:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826275.923836:0:125377:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:00080000:3.0:1787826275.923867:0:125377:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00000100:00100000:3.0:1787826275.923919:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524059c0 going to sleep for 11 seconds 00000001:00080000:2.0:1787826275.930630:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985325 is committed 00000020:00080000:2.0:1787826275.930651:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7855667b40 refcount 2 committed 0 result 0 batchid 3389 00000020:00080000:2.0:1787826275.930655:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2a00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7855667a80 00000020:00080000:2.0:1787826275.930659:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1491 00000020:00080000:2.0:1787826275.930661:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7849231200 00000020:00080000:2.0:1787826275.930664:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1173 00000020:00080000:2.0:1787826275.930680:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3387 batchid updated 00000001:00080000:2.0:1787826275.930685:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985322 is committed 00000020:00080000:2.0:1787826275.930688:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7848d5e240 refcount 2 committed 0 result 0 batchid 3388 00000020:00080000:2.0:1787826275.930690:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952e3c500 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7848d5ed80 00000020:00080000:2.0:1787826275.930693:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1490 00000020:00080000:2.0:1787826275.930695:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952e3c900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7847b1fa00 00000020:00080000:2.0:1787826275.930697:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1172 00000001:00080000:2.0:1787826275.930706:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985320 is committed 00010000:00010000:0.0:1787826275.930774:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930785:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930792:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930798:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.930803:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930810:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930826:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:0.0:1787826275.930829:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.930834:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930840:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f49420 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930853:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930859:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930865:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930869:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.930872:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930877:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.930884:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:0.0:1787826275.930886:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.932993:0:3639:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f495bd expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933033:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933041:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49427 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933071:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933078:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933085:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933091:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.933095:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933101:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933131:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x4ec94fd6 (ffff9f794b57a000) 00010000:00010000:0.0:1787826275.933134:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.933137:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.933143:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f494c8 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787826275.933956:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3389 committed batchid 3387 00000004:00080000:1.0:1787826275.934142:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81c00 with version 6780 00000020:00080000:1.0:1787826275.934158:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3386 cancel update log [0x200000401:0x3:0x0].1488: rc = 0 00000020:00080000:1.0:1787826275.934203:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3386 cancel update log [0x240000400:0x3:0x0].1170: rc = 0 00000004:00080000:1.0:1787826275.934230:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81f00 with version 6781 00000020:00080000:1.0:1787826275.934233:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3387 cancel update log [0x200000401:0x3:0x0].1489: rc = 0 00000020:00080000:1.0:1787826275.934254:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3387 cancel update log [0x240000400:0x3:0x0].1171: rc = 0 00000020:00080000:1.0:1787826275.934277:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3387 00000004:00080000:0.0:1787826275.934708:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6780 rpc_version 6780 00000020:00100000:0.0:1787826275.934742:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826275.934752:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b19ea00 x1874663346107520/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826275.934775:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346107520:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826275.934846:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.934884:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826275.934944:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346107520 00000100:00100000:2.0:1787826275.934952:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346107520 00000100:00100000:2.0:1787826275.935000:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23267 00000100:00100000:2.0:1787826275.935026:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346107520:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826275.935194:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346107520:12345-0@lo:1000:osp_up0-1.0 Request processed in 184us (347us total) trans 8589957309 rc 0/0 00000100:00100000:2.0:1787826275.935202:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23267 00000100:00100000:0.0:1787826275.935640:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346107520:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826275.935653:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6781 rpc_version 6781 00000020:00100000:0.0:1787826275.935666:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826275.935672:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d816680 x1874663346107776/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826275.935730:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346107776:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826275.935761:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.935789:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826275.938304:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346107776 00000100:00100000:2.0:1787826275.938446:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346107776 00000100:00100000:2.0:1787826275.938510:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23268 00000100:00100000:2.0:1787826275.938521:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346107776:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826275.941572:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346107776:12345-0@lo:1000:osp_up0-1.0 Request processed in 3042us (5798us total) trans 8589957310 rc 0/0 00000100:00100000:2.0:1787826275.941590:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23268 00000100:00100000:1.0:1787826275.942379:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826275.942411:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826275.942437:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332520064 00000100:00100000:1.0:1787826275.942445:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332520064 00010000:00010000:1.0:1787826275.942484:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 7 pid: 8752 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826275.942514:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21840 00000100:00100000:1.0:1787826275.942524:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332520064:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826275.942537:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826275.942544:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 7 pid: 8752 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.942569:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 7 pid: 8752 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.942590:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:1.0:1787826275.942597:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f495bd expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.942608:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f495bd expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.942616:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f495bd expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.942649:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f495bd expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826275.942663:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:17812:1874663346107904:0@lo:105: 00000100:00100000:1.0:1787826275.942705:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826275.942739:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826275.942757:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332520192 00000100:00100000:1.0:1787826275.942761:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332520192 00000100:00100000:1.0:1787826275.942799:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30776 00000100:00100000:1.0:1787826275.942807:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332520192:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826275.942821:0:35352:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826275.942836:0:35352:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f495d9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f08 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826275.942874:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346108032:192.168.203.56@tcp:106: 00000100:00100000:1.0:1787826275.942937:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826275.942960:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346107904 00000100:00100000:1.0:1787826275.942964:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346107904 00000100:00100000:1.0:1787826275.942992:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7807 00000100:00100000:1.0:1787826275.942998:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:17812:x1874663346107904:12345-0@lo:105: 00010000:00010000:1.0:1787826275.943009:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.943018:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.943026:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.943034:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.943041:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.943050:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.943062:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43c00) 00000100:00100000:1.0:1787826275.943442:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:17812:x1874663346107904:12345-0@lo:105: Request processed in 442us (735us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.943450:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7807 00000100:00080000:1.0:1787826275.943481:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826275.943509:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:17812:1874663346107904:0@lo:105: 00010000:00010000:1.0:1787826275.943525:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826275.943680:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332520064:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1157us (1305us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.943688:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21840 00010000:00010000:3.0:1787826275.944359:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f49523 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed0 expref: 6 pid: 8752 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826275.944380:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346107264:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826275.944402:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f495bd expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.944411:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:3.0:1787826275.944414:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826275.944417:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826275.944479:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346107136:12345-0@lo:101:mdt_io00_003.0 Request processed in 21112us (22714us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.944487:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23266 00000100:00100000:1.0:1787826275.944815:0:125380:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985320 gen 2 00000004:00080000:1.0:1787826275.944821:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985320 committed transno 4294985320 00000020:00080000:1.0:1787826275.944827:0:125380:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7844e81000 refcount 2 committed 0 result 0 batchid 4294979075 00000020:00080000:1.0:1787826275.944831:0:125380:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796efc5000 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e813c0 00000020:00080000:1.0:1787826275.944835:0:125380:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].931 00000020:00080000:1.0:1787826275.944837:0:125380:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7944e37b00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784771aa00 00000020:00080000:1.0:1787826275.944839:0:125380:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].928 00000100:00100000:1.0:1787826275.944862:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346107136:0@lo:101:mdt_io00_003.0 00010000:00010000:1.0:1787826275.944878:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826275.944905:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18e:0x0] 00010000:00010000:1.0:1787826275.944944:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.944952:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.944961:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826275.944977:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108160:0@lo:101:mdt_io00_003.0 00000100:00100000:1.0:1787826275.945020:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.945084:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346107776:0@lo:1000:osp_up0-1.0 00000020:00080000:3.0:1787826275.945085:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979075 committed batchid 4294979074 00000020:00080000:3.0:1787826275.945176:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979074 00000100:00100000:1.0:1787826275.945478:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e813c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826275.945624:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346108160 00000100:00100000:2.0:1787826275.945631:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346108160 00000100:00100000:2.0:1787826275.945718:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23269 00000100:00100000:2.0:1787826275.945728:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346108160:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:2.0:1787826275.945743:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826275.945761:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f495e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f495e0 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.945784:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f495e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f495e0 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.945797:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f495e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f495e0 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.945805:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x4ec94fd6 (ffff9f79785cd900) 00010000:00010000:2.0:1787826275.945808:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826275.945811:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826275.945863:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108160:12345-0@lo:101:mdt_io00_003.0 Request processed in 137us (843us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.945870:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23269 00000100:00100000:0.0:1787826275.945936:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108160:0@lo:101:mdt_io00_003.0 00010000:00010000:0.0:1787826275.945949:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f495e7 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.945960:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f495e7 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826275.945975:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18e:0x0] 00010000:00010000:0.0:1787826275.945991:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.946003:0:125380:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.946030:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.946055:0:125380:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.946067:0:125380:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 10 pid: 35094 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826275.946078:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346108288:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826275.946440:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00002000:00010000:2.0:1787826275.949765:0:35352:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000401:0x51c:0x0] updating lvb size: 0 -> 6 00002000:00010000:2.0:1787826275.949775:0:35352:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000401:0x51c:0x0] updating lvb mtime: 0 -> 1787826270 00002000:00010000:2.0:1787826275.949778:0:35352:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000401:0x51c:0x0] updating lvb atime: 0 -> 1787826270 00002000:00010000:2.0:1787826275.949780:0:35352:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000401:0x51c:0x0] updating lvb ctime: 0 -> 1787826270 00002000:00010000:2.0:1787826275.949782:0:35352:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000401:0x51c:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:2.0:1787826275.949808:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346108032:192.168.203.56@tcp:106: 00010000:00010000:2.0:1787826275.949848:0:35352:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f495d9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f08 expref: 6 pid: 35352 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.949871:0:35352:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f495d9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f08 expref: 6 pid: 35352 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.949883:0:35352:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:2.0:1787826275.949956:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332520192:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7150us (7543us total) trans 0 rc 301/301 00000100:00100000:2.0:1787826275.949966:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30776 00000100:00100000:0.0:1787826275.956824:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346108288:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826275.956857:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826275.957372:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826275.957508:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332520320 00000100:00100000:2.0:1787826275.957517:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332520320 00010000:00010000:2.0:1787826275.957565:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 11 pid: 35094 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826275.957605:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21841 00000100:00100000:2.0:1787826275.957615:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332520320:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826275.957629:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826275.957636:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 11 pid: 35094 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.957649:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 11 pid: 35094 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.957670:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49531 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ed7 expref: 11 pid: 35094 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.957691:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:2.0:1787826275.957696:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.957704:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.957711:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.957739:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826275.957789:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332520320:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 175us (419us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.957797:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21841 00010000:00010000:0.0:1787826275.957926:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.957935:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.957957:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.957972:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.957980:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826275.958052:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108416:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.958148:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.958186:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826275.958659:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346108416 00000100:00100000:2.0:1787826275.958666:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346108416 00000100:00100000:2.0:1787826275.958713:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23270 00000100:00100000:2.0:1787826275.958722:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108416:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.958809:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108416:12345-0@lo:1000:mdt_io00_003.0 Request processed in 87us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.958816:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23270 00000100:00100000:0.0:1787826275.958960:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108416:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.959020:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108544:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.959063:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.959097:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826275.959248:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826275.960154:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346108544 00000100:00100000:2.0:1787826275.960173:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346108544 00000100:00100000:2.0:1787826275.960223:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23271 00000100:00100000:2.0:1787826275.960232:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108544:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.960348:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108544:12345-0@lo:1000:mdt_io00_003.0 Request processed in 117us (1284us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.960359:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23271 00000100:00100000:0.0:1787826275.961168:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108544:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.961239:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91df80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108672:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.961292:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.961327:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826275.961931:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346108672 00000100:00100000:2.0:1787826275.961940:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346108672 00000100:00100000:2.0:1787826275.961985:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23272 00000100:00100000:2.0:1787826275.961995:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108672:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.962195:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108672:12345-0@lo:1000:mdt_io00_003.0 Request processed in 201us (902us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826275.962202:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23272 00000100:00100000:1.0:1787826275.962299:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91df80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108672:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.962385:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108800:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.962429:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826275.962453:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826275.963215:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346108800 00000100:00100000:2.0:1787826275.963223:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346108800 00000100:00100000:2.0:1787826275.963275:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23273 00000100:00100000:2.0:1787826275.963284:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108800:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.963458:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108800:12345-0@lo:1000:mdt_io00_003.0 Request processed in 174us (1028us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.963465:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23273 00000100:00100000:1.0:1787826275.963812:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108800:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.963913:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108928:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.963946:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826275.963975:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826275.964084:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346108928 00000100:00100000:0.0:1787826275.964442:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346108928 00000100:00100000:0.0:1787826275.964479:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23274 00000100:00100000:0.0:1787826275.964488:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108928:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.964732:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346108928:12345-0@lo:1000:mdt_io00_003.0 Request processed in 243us (784us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826275.964740:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23274 00000100:00100000:1.0:1787826275.964811:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346108928:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826275.964843:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e40:0x0] 00010000:00010000:1.0:1787826275.964942:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.964960:0:125380:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.964969:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.964995:0:125380:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.965010:0:125380:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 9 pid: 6528 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826275.965021:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346109056:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826275.965125:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.965814:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332520448 00000100:00100000:3.0:1787826275.965822:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332520448 00000100:00100000:3.0:1787826275.965863:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30777 00000100:00100000:3.0:1787826275.965872:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:223574:x1874663332520448:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826275.965885:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826275.965912:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f0f expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.965939:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.965947:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x51c:0x0].0x0 since=5s v=(0 0) 00010000:00010000:3.0:1787826275.965951:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826275.965956:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.965981:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.965991:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 5 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826275.966002:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346109184:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826275.966070:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524059c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826275.975422:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826275.975457:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826275.975490:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332520576 00000100:00100000:1.0:1787826275.975499:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332520576 00010000:00010000:1.0:1787826275.975542:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 10 pid: 6528 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826275.975589:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21842 00000100:00100000:1.0:1787826275.975600:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332520576:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826275.975616:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826275.975624:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 10 pid: 6528 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.975639:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 10 pid: 6528 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.975667:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f784abfe800) 00000100:00100000:3.0:1787826275.975670:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346109184:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826275.975673:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.975688:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.975696:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.975713:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826275.975718:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.975737:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:1.0:1787826275.975761:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332520576:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 161us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.975769:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21842 00000100:00100000:3.0:1787826275.975774:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223574:x1874663332520448:12345-192.168.203.56@tcp:101:cat.0 Request processed in 9899us (16524us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.975783:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30777 00010000:00010000:2.0:1787826275.975969:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f495c4 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ede expref: 9 pid: 6528 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826275.975986:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346109056:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826275.976002:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.976010:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.976016:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.976073:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.976081:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.976098:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826275.976156:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109312:0@lo:101:mdt_io00_003.0 00000100:00100000:2.0:1787826275.976194:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826275.976218:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.976329:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346109312 00000100:00100000:3.0:1787826275.976335:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346109312 00000100:00100000:0.0:1787826275.976343:0:20169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332520704 00000100:00100000:0.0:1787826275.976349:0:20169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332520704 00000100:00100000:3.0:1787826275.976386:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23275 00000100:00100000:3.0:1787826275.976400:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346109312:12345-0@lo:101:mdt_io00_003.0 00002000:00100000:0.0:1787826275.976401:0:20169:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x51c:0x0] (0->5) req@ffff9f784b19c380 x1874663332520704/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:631/0 lens 496/0 e 0 to 0 dl 1787826286 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826275.976414:0:20169:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b19c380 with x1874663332520704 ext(0->5) 00010000:00010000:0.0:1787826275.976421:0:20169:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 6 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.976427:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826275.976435:0:20169:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 6 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826275.976443:0:20169:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9f784b19c380 00010000:00010000:3.0:1787826275.976446:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f49611 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4960a expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826275.976459:0:20169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2487 00000100:00100000:0.0:1787826275.976466:0:20169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332520704:12345-192.168.203.56@tcp:4:bash.0 00010000:00010000:3.0:1787826275.976472:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f49611 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4960a expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.976485:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f49611 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4960a expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826275.976493:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:3.0:1787826275.976496:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1787826275.976499:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:3.0:1787826275.976536:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109312:12345-0@lo:101:mdt_io00_003.0 Request processed in 142us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.976542:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23275 00000100:00100000:2.0:1787826275.976627:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109312:0@lo:101:mdt_io00_003.0 00010000:00010000:2.0:1787826275.976639:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49611 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.976649:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49611 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826275.976663:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e41:0x0] 00000100:00100000:2.0:1787826275.976699:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109440:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.976723:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826275.976747:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548ae40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.977035:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346109440 00000100:00100000:3.0:1787826275.977040:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346109440 00000100:00100000:3.0:1787826275.977074:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23276 00000100:00100000:3.0:1787826275.977082:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109440:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.977167:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109440:12345-0@lo:1000:mdt_io00_003.0 Request processed in 86us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.977174:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23276 00000100:00080000:1.0:1787826275.978068:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826275.978154:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109440:0@lo:1000:mdt_io00_003.0 00000004:00080000:3.0:1787826275.978193:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18e:0x0] 00000100:00100000:3.0:1787826275.978237:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109568:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.978286:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826275.978316:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826275.978547:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346109568 00000100:00100000:1.0:1787826275.978563:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346109568 00000100:00100000:1.0:1787826275.978612:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23277 00000100:00100000:1.0:1787826275.978621:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109568:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.978708:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109568:12345-0@lo:1000:mdt_io00_003.0 Request processed in 87us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.978715:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23277 00000100:00100000:1.0:1787826275.978745:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109568:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826275.978764:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e40:0x0] 00010000:00010000:1.0:1787826275.978800:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.978818:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.978828:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826275.978875:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaad80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109696:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.978916:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826275.978940:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346109696 00000100:00100000:1.0:1787826275.978944:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346109696 00000100:00100000:1.0:1787826275.978968:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23278 00000100:00100000:1.0:1787826275.978974:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109696:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.979322:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109696:12345-0@lo:1000:mdt_io00_003.0 Request processed in 346us (404us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.979329:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23278 00000100:00100000:1.0:1787826275.979349:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826275.979362:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaad80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109696:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826275.979386:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e40:0x0] 00000100:00100000:1.0:1787826275.979408:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaad80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109824:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.979432:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826275.979456:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.979504:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346109824 00000100:00100000:3.0:1787826275.979509:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346109824 00000100:00100000:3.0:1787826275.979543:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23279 00000100:00100000:3.0:1787826275.979559:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109824:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.979611:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109824:12345-0@lo:1000:mdt_io00_003.0 Request processed in 53us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.979617:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23279 00000100:00100000:1.0:1787826275.980325:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaad80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109824:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.980380:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaad80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109952:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.980414:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826275.980440:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.980502:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346109952 00000100:00100000:3.0:1787826275.980508:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346109952 00000100:00100000:3.0:1787826275.980937:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23280 00000100:00100000:3.0:1787826275.980951:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109952:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.981056:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346109952:12345-0@lo:1000:mdt_io00_003.0 Request processed in 108us (641us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.981131:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaad80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346109952:0@lo:1000:mdt_io00_003.0 00000004:00080000:2.0:1787826275.981161:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18e:0x0] 00000100:00100000:2.0:1787826275.981195:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cff100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346110080:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.981228:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826275.981248:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548af00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826275.984793:0:20169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332520704:12345-192.168.203.56@tcp:4:bash.0 Request processed in 8316us (9368us total) trans 4294973835 rc 0/0 00000100:00100000:2.0:1787826275.984808:0:20169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2487 00002000:00100000:2.0:1787826275.984821:0:20169:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x51c:0x0] (0->5) req@ffff9f784b19c380 x1874663332520704/t4294973835(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:631/0 lens 496/448 e 0 to 0 dl 1787826286 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826275.984832:0:20169:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b19c380 with x1874663332520704 ext(0->5) 00010000:00010000:2.0:1787826275.984843:0:20169:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 8 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.984861:0:20169:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 8 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826275.984869:0:20169:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9f784b19c380 00000001:00080000:2.0:1787826275.985762:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973834 is committed 00000100:00100000:1.0:1787826275.985937:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346110080 00000100:00100000:1.0:1787826275.985946:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346110080 00000100:00100000:1.0:1787826275.985995:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23281 00000100:00100000:1.0:1787826275.986005:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:125380:x1874663346110080:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.986388:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:125380:x1874663346110080:12345-0@lo:1000:mdt_io00_003.0 Request processed in 382us (5157us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.986395:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23281 00000100:00100000:3.0:1787826275.986463:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23280 00000100:00100000:2.0:1787826275.986548:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cff100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346110080:0@lo:1000:mdt_io00_003.0 00000004:00080000:2.0:1787826275.986647:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e40:0x0] 00000100:00100000:2.0:1787826275.986697:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346110208:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.986729:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826275.986756:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826275.986798:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346110208 00000100:00100000:1.0:1787826275.986802:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346110208 00000100:00100000:1.0:1787826275.986834:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23282 00000100:00100000:1.0:1787826275.986841:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346110208:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826275.986947:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346110208:12345-0@lo:1000:mdt_io00_003.0 Request processed in 106us (217us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.986954:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23282 00000100:00100000:3.0:1787826275.988618:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346110208:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.988846:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346110336:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.988913:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826275.988985:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548ae40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.989325:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346110336 00000100:00100000:3.0:1787826275.989333:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346110336 00000100:00100000:2.0:1787826275.989483:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23283 00000100:00100000:2.0:1787826275.989494:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346110336:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826275.989594:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346110336:12345-0@lo:1000:mdt_io00_003.0 Request processed in 101us (680us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826275.989602:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23283 00000100:00100000:0.0:1787826275.989668:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346110336:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.989756:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346110464:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826275.989785:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826275.989969:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826275.990257:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346110464 00000100:00100000:3.0:1787826275.990263:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346110464 00000100:00100000:3.0:1787826275.990309:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23284 00000100:00100000:3.0:1787826275.990318:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346110464:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826275.990387:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346110464:12345-0@lo:1000:mdt_io00_003.0 Request processed in 71us (601us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826275.990394:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23284 00000100:00100000:0.0:1787826275.990531:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346110464:0@lo:1000:mdt_io00_003.0 00000004:00080000:0.0:1787826275.990902:0:125380:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1db4:0x0] is inserted into orphan list, open count = 1 00000040:00080000:0.0:1787826275.990958:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].929, 1240 off873912 00000040:00080000:0.0:1787826275.991004:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].932, 1320 off972576 00000004:00080000:0.0:1787826275.991009:0:125380:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7846153840 with version 1812 00000004:00080000:3.0:1787826275.991328:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1812 rpc_version 1812 00000020:00100000:3.0:1787826275.991354:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826275.991361:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b304e00 x1874663346110720/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826275.991381:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346110720:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826275.991424:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826275.991454:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826275.991512:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346110720 00000100:00100000:0.0:1787826275.991518:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346110720 00000100:00100000:0.0:1787826275.991668:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23285 00000100:00100000:0.0:1787826275.991678:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346110720:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826275.991924:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346110720:12345-0@lo:1000:osp_up1-0.0 Request processed in 246us (498us total) trans 4294985326 rc 0/0 00000100:00100000:0.0:1787826275.991931:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23285 00000100:00100000:3.0:1787826275.991996:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346110720:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787826275.992377:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992387:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992396:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992402:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.992407:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992415:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992429:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f784abfe800) 00010000:00010000:0.0:1787826275.992431:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826275.992435:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992441:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f495fc lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826275.992466:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e41:0x0] 00010000:00010000:0.0:1787826275.992472:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49611 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992485:0:125380:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49611 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992497:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992503:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992510:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992515:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.992518:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992524:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992533:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:0.0:1787826275.992535:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.992538:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992543:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f495ee lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826275.992637:0:125380:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006b746b17 reply state 00000000b8015a21 transno 8589957311 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992655:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992662:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f79460a2e00) 00010000:00010000:0.0:1787826275.992664:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826275.992666:0:125380:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:0.0:1787826275.992670:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18e:0x0] 00010000:00010000:0.0:1787826275.992674:0:125380:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992681:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992688:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.992995:0:125380:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826275.993003:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18e:0x0] 00010000:00010000:0.0:1787826275.993017:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f495e7 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.993270:0:125380:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f495e7 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.993282:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.993289:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.993295:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.993300:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826275.993303:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.993310:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.993320:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7845cb1800) 00010000:00010000:0.0:1787826275.993322:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826275.993326:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.993331:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49618 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826275.993423:0:125380:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:223576:x1874663332519936:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 83197us (83417us total) trans 8589957311 rc 0/0 00000100:00100000:0.0:1787826275.993432:0:125380:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1223 00010000:00010000:2.0:1787826275.994353:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826275.994364:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.994374:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826275.994380:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826275.994388:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.994535:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.994542:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826275.994643:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346110848:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826275.994698:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826275.994769:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346110848 00000100:00100000:1.0:1787826275.994776:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346110848 00000100:00100000:1.0:1787826275.994833:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10991 00000100:00100000:1.0:1787826275.994844:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346110848:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826275.994857:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826275.994875:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f495d2 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f495bd expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826275.994891:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7961d81200) 00010000:00010000:1.0:1787826275.994913:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826275.995004:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 1/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.995020:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f495bd lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f495d2 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826275.995045:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826275.995047:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826275.995049:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826275.995440:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3655:x1874663346110848:12345-0@lo:103:ldlm_bl.0 Request processed in 597us (741us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826275.995448:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10991 00000100:00100000:2.0:1787826275.995617:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346110848:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826275.996876:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826275.997059:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332520832 00000100:00100000:2.0:1787826275.997068:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332520832 00010000:00010000:2.0:1787826275.997136:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 8 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826275.997290:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21843 00000100:00100000:2.0:1787826275.997301:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332520832:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826275.997316:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1787826275.997334:0:6513:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x51c:0x0] updating lvb blocks from disk: 1 -> 8 00010000:00010000:2.0:1787826275.997340:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 8 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.997353:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 8 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.997376:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f494b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ea6 expref: 8 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.997394:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826275.997398:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826275.997408:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.997429:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826275.997466:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826275.997479:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6513:1874663346110976:192.168.203.56@tcp:105: 00000100:00100000:2.0:1787826275.997912:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851d70600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826276.007997:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6513:1874663346110976:192.168.203.56@tcp:105: 00010000:00010000:2.0:1787826276.008033:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826276.008083:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332520832:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 10783us (11214us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826276.008099:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21843 00000100:00100000:1.0:1787826276.014867:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826276.014974:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332520960 00000100:00100000:2.0:1787826276.014985:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332520960 00000100:00100000:2.0:1787826276.015056:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68214 00000100:00100000:2.0:1787826276.015068:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223574:x1874663332520960:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826276.015091:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826276.015141:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4961f lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1db4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f16 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.015204:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4961f lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1db4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f16 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.015217:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4961f lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1db4:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f16 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.015233:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f7952ca9500) 00010000:00010000:2.0:1787826276.015236:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787826276.015239:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826276.015285:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223574:x1874663332520960:12345-192.168.203.56@tcp:101:cat.0 Request processed in 220us (430us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826276.015291:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68214 00000100:00100000:1.0:1787826276.024943:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826276.025145:0:38010:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332521088 00000100:00100000:3.0:1787826276.025155:0:38010:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332521088 00002000:00100000:3.0:1787826276.025215:0:38010:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_013: refresh rw locks for [0x280000401:0x51c:0x0] (0->4095) req@ffff9f7977e0d500 x1874663332521088/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:632/0 lens 488/0 e 0 to 0 dl 1787826287 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826276.025226:0:38010:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977e0d500 with x1874663332521088 ext(0->4095) 00002000:00010000:3.0:1787826276.025231:0:38010:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7977e0d500 00000100:00100000:3.0:1787826276.025266:0:38010:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27630 00000100:00100000:3.0:1787826276.025276:0:38010:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332521088:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:3.0:1787826276.025685:0:38010:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332521088:12345-192.168.203.56@tcp:3:cat.0 Request processed in 409us (749us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787826276.025693:0:38010:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27630 00002000:00100000:3.0:1787826276.025700:0:38010:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_013: refresh rw locks for [0x280000401:0x51c:0x0] (0->4095) req@ffff9f7977e0d500 x1874663332521088/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:632/0 lens 488/4536 e 0 to 0 dl 1787826287 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826276.025708:0:38010:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977e0d500 with x1874663332521088 ext(0->4095) 00002000:00010000:3.0:1787826276.025711:0:38010:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7977e0d500 00000100:00100000:1.0:1787826276.032474:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826276.032742:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332521344 00000100:00100000:2.0:1787826276.032752:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332521344 00000100:00100000:2.0:1787826276.032810:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9298 00000100:00100000:2.0:1787826276.032820:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223574:x1874663332521344:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:2.0:1787826276.033263:0:17601:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1db4:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1787826276.033332:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].210, 64 off21632 00000004:00080000:2.0:1787826276.033345:0:17601:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8088245238317ns, next at 8089245235317ns, rc = 0 00010000:00010000:2.0:1787826276.033399:0:17601:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.033416:0:17601:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.033428:0:17601:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.033437:0:17601:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.033444:0:17601:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.033449:0:17601:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.033465:0:17601:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826276.033700:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826276.033708:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826276.033716:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826276.033720:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826276.033727:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826276.033740:0:30445:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f7952ca9500) 00010000:00010000:0.0:1787826276.033744:0:30445:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826276.033748:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826276.033754:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49626 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db4:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826276.033768:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826276.033770:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826276.033771:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826276.034001:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223574:x1874663332521344:12345-192.168.203.56@tcp:35:cat.0 Request processed in 1179us (1532us total) trans 8589957312 rc 0/0 00000100:00100000:2.0:1787826276.034011:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9298 00010000:00010000:3.0:1787826276.400953:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f495e7 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826276.400979:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f495e7 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.401032:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f495e7 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.401040:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826276.401148:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346111104:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826276.401233:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826276.401378:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece400/0xe21309aa81f495e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f495e7 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.401399:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49611 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826276.401407:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49611 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.401426:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49611 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.401433:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826276.401481:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346111232:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826276.401512:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826276.401564:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346111104 00000100:00100000:0.0:1787826276.401571:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346111104 00000100:00100000:0.0:1787826276.401616:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346111232 00000100:00100000:0.0:1787826276.401620:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346111232 00000100:00100000:0.0:1787826276.401644:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10992 00000100:00100000:0.0:1787826276.401653:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c81450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+9:3652:x1874663346111104:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826276.401667:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826276.401686:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f495e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18e:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f495e0 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826276.401701:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x4ec94fd6 (ffff9f79785cd900) 00010000:00010000:0.0:1787826276.401712:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826276.401780:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10993 00000100:00100000:2.0:1787826276.401792:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346111232:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826276.401801:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826276.401813:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f49611 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4960a expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.401830:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:2.0:1787826276.401840:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826276.401872:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346111232:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 83us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826276.401879:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10993 00000100:00100000:2.0:1787826276.404229:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346111232:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826276.404984:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4960a lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49611 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826276.405016:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957312 is committed 00000004:00080000:3.0:1787826276.405025:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 209 committed 00000001:00080000:3.0:1787826276.405068:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957311 is committed 00000020:00080000:3.0:1787826276.405078:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979075 batchid updated 00000001:00080000:3.0:1787826276.405088:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957310 is committed 00000020:00080000:3.0:1787826276.405127:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979075 00000004:00080000:3.0:1787826276.405202:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7852405900 with version 1813 00000020:00080000:3.0:1787826276.405212:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979075 cancel update log [0x240000404:0x2:0x0].931: rc = 0 00000020:00080000:3.0:1787826276.405302:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979075 cancel update log [0x200000bd0:0x2:0x0].928: rc = 0 00000020:00080000:3.0:1787826276.405311:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979075 00000004:00080000:3.0:1787826276.405320:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1813 rpc_version 1813 00000020:00100000:3.0:1787826276.405349:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826276.405358:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b304700 x1874663346111488/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826276.408576:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346111488:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826276.408666:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826276.408699:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826276.408731:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346111488 00000100:00100000:3.0:1787826276.408738:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346111488 00000100:00100000:3.0:1787826276.408789:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23286 00000100:00100000:3.0:1787826276.408799:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346111488:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826276.409080:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346111488:12345-0@lo:1000:osp_up1-0.0 Request processed in 281us (413us total) trans 4294985327 rc 0/0 00000100:00100000:3.0:1787826276.409089:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23286 00000100:00100000:3.0:1787826276.409348:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346111488:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787826276.409397:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409409:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409417:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409424:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826276.409428:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409436:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409452:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826276.409455:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826276.409460:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409467:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f495f5 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409518:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409525:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409531:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409536:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826276.409539:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409545:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409554:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826276.409556:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826276.409559:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.409564:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4949e lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826276.409769:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826276.409780:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346111616:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787826276.409826:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826276.409922:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346111616 00000100:00100000:2.0:1787826276.409928:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346111616 00000100:00100000:2.0:1787826276.410019:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30778 00000100:00100000:2.0:1787826276.410028:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346111616:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1787826276.410091:0:35097:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1308 count 1 00010000:00010000:2.0:1787826276.410174:0:35097:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.410227:0:35097:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.410255:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826276.410260:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826276.410265:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 4/0,1 mode: --/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.410839:0:35097:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.410856:0:35097:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 5 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826276.410870:0:35097:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:35097:1874663346111744:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826276.410955:0:35097:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f796ed379c0 going to sleep for 11 seconds 00000100:00080000:2.0:1787826276.411275:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826276.414796:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826276.414865:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332521472 00000100:00100000:3.0:1787826276.414875:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332521472 00010000:00010000:3.0:1787826276.414924:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 6 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826276.414958:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21844 00000100:00100000:3.0:1787826276.414968:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332521472:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826276.414981:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826276.414987:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 6 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.415001:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 6 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.415037:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826276.415042:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826276.415049:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.415058:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.415078:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826276.415152:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332521472:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 184us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826276.415169:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21844 00010000:00010000:2.0:1787826276.416419:0:35097:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392ed720/0xe21309aa81f49603 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f0f expref: 5 pid: 17826 timeout: 8188 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826276.416444:0:35097:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:35097:1874663346111744:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826276.416466:0:35097:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.416477:0:35097:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.416495:0:35097:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.416506:0:35097:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.416531:0:35097:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826276.416570:0:35097:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826276.416652:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826276.416659:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.416668:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826276.416672:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.416680:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.416702:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 1/0,0 mode: --/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.416709:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4962d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826276.416737:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826276.416740:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826276.416742:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826276.417087:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346111616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7047us (7247us total) trans 4294973836 rc 0/0 00000100:00100000:2.0:1787826276.417098:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30778 00000004:00080000:3.0:1787826276.417134:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784b759c00/2, rc 0, transno 4294973836 00000100:00100000:3.0:1787826276.417141:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346111616:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1787826276.421838:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c81450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346111104:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 20174us (20593us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826276.421875:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10992 00000100:00100000:1.0:1787826276.422846:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346111104:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826276.530872:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826276.574466:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826276.574550:0:24220:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332521600 00000100:00100000:3.0:1787826276.574561:0:24220:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332521600 00000100:00080000:1.0:1787826276.574630:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00100000:3.0:1787826276.574638:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000400:0x4c7:0x0] (0->5) req@ffff9f784a91fb80 x1874663332521600/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:632/0 lens 496/0 e 0 to 0 dl 1787826287 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826276.574650:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784a91fb80 with x1874663332521600 ext(0->5) 00002000:00010000:3.0:1787826276.574656:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784a91fb80 00000100:00100000:3.0:1787826276.574689:0:24220:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27631 00000100:00100000:3.0:1787826276.574701:0:24220:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332521600:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826276.575589:0:24220:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332521600:12345-192.168.203.56@tcp:4:bash.0 Request processed in 885us (1125us total) trans 4294976144 rc 0/0 00000100:00100000:3.0:1787826276.575599:0:24220:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27631 00002000:00100000:3.0:1787826276.575607:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000400:0x4c7:0x0] (0->5) req@ffff9f784a91fb80 x1874663332521600/t4294976144(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:632/0 lens 496/448 e 0 to 0 dl 1787826287 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826276.575620:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784a91fb80 with x1874663332521600 ext(0->5) 00002000:00010000:3.0:1787826276.575625:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784a91fb80 00000001:00080000:0.0:1787826276.577099:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976144 is committed 00000001:00080000:0.0:1787826276.577142:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976143 is committed 00000100:00100000:1.0:1787826276.578988:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826276.579526:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332521728 00000100:00100000:0.0:1787826276.579538:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332521728 00000100:00100000:0.0:1787826276.579655:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30779 00000100:00100000:0.0:1787826276.579667:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332521728:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:0.0:1787826276.579777:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332521728:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 113us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826276.579784:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30779 00000100:00100000:1.0:1787826277.224553:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.224689:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332521856 00000100:00100000:3.0:1787826277.224700:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332521856 00000100:00100000:3.0:1787826277.224779:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68215 00000100:00100000:3.0:1787826277.224792:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223612:x1874663332521856:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826277.224813:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826277.224837:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49634 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f1d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.224924:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4963b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.224938:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4963b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.224953:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4963b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826277.225051:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e40:0x0]: returning size 4096 00000004:00200000:3.0:1787826277.225056:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787826277.225074:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4963b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826277.225287:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1787826277.225433:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49634 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f1d expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.225457:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4963b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f1d expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.225466:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:3.0:1787826277.225469:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826277.225473:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826277.226864:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223612:x1874663332521856:12345-192.168.203.56@tcp:101:rm.0 Request processed in 2066us (2312us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.226877:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68215 00000100:00100000:0.0:1787826277.244224:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.244473:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332521984 00000100:00100000:2.0:1787826277.244482:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332521984 00000100:00100000:2.0:1787826277.244551:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68216 00000100:00100000:2.0:1787826277.244563:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223612:x1874663332521984:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826277.244583:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826277.244607:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49642 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f24 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.244671:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.244684:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.244695:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826277.244744:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18e:0x0]: returning size 4096 00000004:00200000:2.0:1787826277.244749:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826277.244782:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49642 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.244798:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.244822:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:2.0:1787826277.244826:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826277.244830:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826277.244898:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223612:x1874663332521984:12345-192.168.203.56@tcp:101:rm.0 Request processed in 337us (681us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.244906:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68216 00000100:00100000:1.0:1787826277.255306:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.257284:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332522112 00000100:00100000:2.0:1787826277.257298:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332522112 00000100:00100000:2.0:1787826277.257374:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68217 00000100:00100000:2.0:1787826277.257385:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223612:x1874663332522112:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826277.257408:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826277.257432:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49650 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f2b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.257506:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.257521:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.257534:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826277.257579:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18c:0x0]: returning size 4096 00000004:00200000:2.0:1787826277.257585:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826277.257623:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49650 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.257640:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.257650:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:2.0:1787826277.257653:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826277.257657:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826277.257722:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223612:x1874663332522112:12345-192.168.203.56@tcp:101:rm.0 Request processed in 338us (2424us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.257729:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68217 00000100:00080000:1.0:1787826277.259661:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826277.281301:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.281378:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332522368 00000100:00100000:3.0:1787826277.281388:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332522368 00000100:00100000:3.0:1787826277.281451:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68218 00000100:00100000:3.0:1787826277.281462:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223612:x1874663332522368:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826277.281485:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826277.281618:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4965e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f32 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826277.281777:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e40:0x0]: returning size 4096 00010000:00010000:3.0:1787826277.281873:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4965e lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f32 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.281898:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826277.281902:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826277.281908:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4965e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f32 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.281928:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826277.282007:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223612:x1874663332522368:12345-192.168.203.56@tcp:101:rm.0 Request processed in 547us (717us total) trans 4294985328 rc 301/301 00000100:00100000:3.0:1787826277.282015:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68218 00000100:00100000:3.0:1787826277.293200:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826277.294937:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332522496 00000100:00100000:0.0:1787826277.294952:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332522496 00000100:00100000:0.0:1787826277.295016:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9299 00000100:00100000:0.0:1787826277.295028:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223612:x1874663332522496:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826277.307815:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223612:x1874663332522496:12345-192.168.203.56@tcp:37:rm.0 Request processed in 12777us (14614us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.307829:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9299 00000100:00100000:3.0:1787826277.321140:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.321174:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332522624 00000100:00100000:2.0:1787826277.321185:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332522624 00000100:00100000:2.0:1787826277.321249:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9300 00000100:00100000:2.0:1787826277.321261:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943005050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223612:x1874663332522624:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826277.325724:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943005050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223612:x1874663332522624:12345-192.168.203.56@tcp:37:rm.0 Request processed in 4453us (4581us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.325738:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9300 00000100:00100000:3.0:1787826277.336749:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.337318:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332522880 00000100:00100000:2.0:1787826277.337327:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332522880 00000100:00100000:2.0:1787826277.337717:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9301 00000100:00100000:2.0:1787826277.337728:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223612:x1874663332522880:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:2.0:1787826277.337987:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1174, 344 off1067856 00000040:00080000:2.0:1787826277.338036:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1492, 424 off1507232 00000004:00080000:2.0:1787826277.338042:0:6531:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7850703cc0 with version 6782 00000004:00080000:1.0:1787826277.339935:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6782 rpc_version 6782 00000020:00100000:1.0:1787826277.339975:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826277.339985:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977e0fb80 x1874663346112000/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826277.340009:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346112000:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826277.340237:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.340268:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.340531:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346112000 00000100:00100000:0.0:1787826277.340539:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346112000 00000100:00100000:0.0:1787826277.340777:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23287 00000100:00100000:0.0:1787826277.340790:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346112000:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826277.341017:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346112000:12345-0@lo:1000:osp_up0-1.0 Request processed in 230us (779us total) trans 8589957313 rc 0/0 00000100:00100000:3.0:1787826277.341066:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957310 gen 2 00000004:00080000:3.0:1787826277.341073:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957309 committed transno 8589957310 00000004:00080000:3.0:1787826277.341089:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957310 committed transno 8589957310 00000100:00100000:3.0:1787826277.341101:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f7977e0fb80 x1874663346112000/t8589957313(8589957313) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826293 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826277.341150:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346112000:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826277.341252:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223612:x1874663332522880:12345-192.168.203.56@tcp:35:rm.0 Request processed in 3525us (4510us total) trans 4294985329 rc 0/0 00000100:00100000:3.0:1787826277.341261:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9301 00000100:00100000:0.0:1787826277.347150:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23287 00000100:00100000:3.0:1787826277.353485:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826277.355972:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332523136 00000100:00100000:0.0:1787826277.355986:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332523136 00000100:00100000:0.0:1787826277.356057:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68219 00000100:00100000:0.0:1787826277.356068:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223612:x1874663332523136:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826277.356094:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826277.356143:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4966c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f39 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.356294:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49673 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.356308:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49673 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.356320:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49673 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826277.356332:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e40:0x0]: returning size 4096 00010000:00010000:0.0:1787826277.356422:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4966c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f39 expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.356449:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49673 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f39 expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.356458:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:0.0:1787826277.356461:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787826277.356465:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826277.356534:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223612:x1874663332523136:12345-192.168.203.56@tcp:101:rm.0 Request processed in 467us (3258us total) trans 4294985330 rc 0/0 00000100:00100000:0.0:1787826277.356542:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68219 00000100:00100000:3.0:1787826277.363853:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.363942:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332523264 00000100:00100000:1.0:1787826277.363954:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332523264 00000100:00100000:1.0:1787826277.364016:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68220 00000100:00100000:1.0:1787826277.364027:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223612:x1874663332523264:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:1.0:1787826277.364078:0:8752:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345651712 00000100:00100000:1.0:1787826277.364164:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223612:x1874663332523264:12345-192.168.203.56@tcp:41:rm.0 Request processed in 139us (321us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.364173:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68220 00000100:00100000:3.0:1787826277.366584:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826277.367727:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332523392 00000100:00100000:0.0:1787826277.367741:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332523392 00000100:00100000:0.0:1787826277.367835:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68221 00000100:00100000:0.0:1787826277.367847:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223612:x1874663332523392:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826277.367866:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826277.367897:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49681 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f40 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.367965:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.367979:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.367992:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368009:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368017:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368023:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368080:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49696 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368088:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49696 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368094:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49696 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826277.368142:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49696 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368158:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368165:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368171:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368177:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368183:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826277.368187:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368195:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368207:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:0.0:1787826277.368210:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826277.368215:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368221:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49688 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368234:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368241:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368246:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368251:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826277.368253:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368258:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368265:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f794b57af00) 00010000:00010000:0.0:1787826277.368267:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826277.368270:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368275:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4968f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368310:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49681 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f40 expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368323:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49696 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f40 expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.368330:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f794b57aa00) 00010000:00010000:0.0:1787826277.368332:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826277.368334:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826277.368452:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223612:x1874663332523392:12345-192.168.203.56@tcp:101:rm.0 Request processed in 606us (1874us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.368459:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68221 00000100:00100000:3.0:1787826277.372780:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826277.372867:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332523520 00000100:00100000:0.0:1787826277.372892:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332523520 00000100:00100000:0.0:1787826277.372976:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68222 00000100:00100000:0.0:1787826277.372988:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223612:x1874663332523520:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826277.373015:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826277.373038:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4969d lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f47 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.373139:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f496a4 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.373153:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f496a4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.373165:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f496a4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826277.373210:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e41:0x0]: returning size 6 00000004:00200000:0.0:1787826277.373214:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:0.0:1787826277.373249:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4969d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f47 expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.373270:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f496a4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f47 expref: 13 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.373279:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826277.373282:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826277.373285:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826277.373352:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223612:x1874663332523520:12345-192.168.203.56@tcp:101:rm.0 Request processed in 366us (588us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.373359:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68222 00000100:00100000:2.0:1787826277.381344:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.381394:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.382066:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332523648 00000100:00100000:3.0:1787826277.382078:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332523648 00000100:00100000:3.0:1787826277.382318:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10994 00000100:00100000:3.0:1787826277.382330:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332523648:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826277.382346:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826277.382378:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49696 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f40 expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.382404:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f794b57aa00) 00010000:00010000:3.0:1787826277.382417:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826277.382442:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332523904 00000100:00100000:1.0:1787826277.382448:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332523904 00000100:00100000:3.0:1787826277.382481:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332523648:12345-192.168.203.56@tcp:103:rm.0 Request processed in 152us (1084us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.382488:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10994 00000100:00100000:1.0:1787826277.382528:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68223 00000100:00100000:1.0:1787826277.382537:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223612:x1874663332523904:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826277.382585:0:6529:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826277.382600:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f496a4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e41:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f47 expref: 13 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.382614:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:1.0:1787826277.382622:0:6529:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826277.382668:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.382677:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.382688:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.382710:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0df80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112128:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787826277.382761:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.382793:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826277.383291:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346112128 00000100:00100000:3.0:1787826277.383299:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346112128 00000100:00100000:3.0:1787826277.383576:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23288 00000100:00100000:3.0:1787826277.383587:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346112128:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787826277.384018:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826277.384036:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f496ab expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.384063:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.384074:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f496ab expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.384125:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.384141:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 9 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.384153:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346112256:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826277.384339:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826277.389973:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346112256:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826277.390012:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496ab expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.390034:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:3.0:1787826277.390038:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826277.390042:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496ab expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.390053:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826277.390147:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346112128:12345-0@lo:101:mdt00_002.0 Request processed in 6562us (7385us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.390155:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23288 00000100:00100000:1.0:1787826277.390245:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0df80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112128:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826277.390262:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.390272:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.391326:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.391415:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332524032 00000100:00100000:1.0:1787826277.391424:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332524032 00010000:00010000:1.0:1787826277.391460:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 10 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.391492:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21845 00000100:00100000:1.0:1787826277.391513:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332524032:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826277.391528:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826277.391534:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 10 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.391547:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 10 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.391567:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49657 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f2b expref: 10 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.391582:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:1.0:1787826277.391588:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496ab expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.391598:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f496ab expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.391607:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f496ab expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.391768:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496ab expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.391783:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346112384:0@lo:105: 00000100:00100000:1.0:1787826277.391828:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.391954:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826277.392169:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346112384 00000100:00100000:2.0:1787826277.392174:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346112384 00000100:00100000:2.0:1787826277.392212:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7808 00000100:00100000:2.0:1787826277.392220:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346112384:12345-0@lo:105: 00010000:00010000:2.0:1787826277.392232:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.392243:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.392251:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.392259:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.392267:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.392275:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.392305:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf200) 00000100:00100000:2.0:1787826277.392359:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346112384:12345-0@lo:105: Request processed in 139us (530us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.392365:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7808 00010000:00010000:1.0:1787826277.393031:0:6529:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.393042:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826277.393066:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18c:0x0] 00010000:00010000:1.0:1787826277.393140:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.393148:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.393172:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.393189:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112512:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787826277.393228:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.393248:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e813c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826277.393561:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346112512 00000100:00100000:2.0:1787826277.393569:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346112512 00000100:00100000:2.0:1787826277.393616:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23289 00000100:00100000:2.0:1787826277.393624:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346112512:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787826277.393640:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826277.393658:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f496c0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f496b9 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.393679:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f496c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f496b9 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.393692:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f496c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496b9 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.393699:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f7952ca9e00) 00010000:00010000:2.0:1787826277.393701:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826277.393705:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826277.393753:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346112512:12345-0@lo:101:mdt00_002.0 Request processed in 130us (524us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.393759:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23289 00000100:00100000:0.0:1787826277.393956:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346112384:0@lo:105: 00010000:00010000:0.0:1787826277.393974:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826277.394030:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332524032:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2517us (2706us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.394039:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21845 00000100:00100000:3.0:1787826277.394058:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112512:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787826277.394075:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.394085:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826277.394098:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18c:0x0] 00000100:00100000:2.0:1787826277.395041:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112640:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826277.395095:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.395379:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.395555:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346112640 00000100:00100000:0.0:1787826277.395562:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346112640 00000100:00100000:0.0:1787826277.395809:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23290 00000100:00100000:0.0:1787826277.395822:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346112640:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826277.395940:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346112640:12345-0@lo:1000:mdt00_002.0 Request processed in 121us (843us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.395948:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23290 00000100:00100000:2.0:1787826277.396027:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112640:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826277.396085:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112768:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826277.396129:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.396252:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826277.396278:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346112768 00000100:00100000:3.0:1787826277.396283:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346112768 00000100:00100000:3.0:1787826277.396312:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23291 00000100:00100000:3.0:1787826277.396320:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346112768:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826277.396397:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346112768:12345-0@lo:1000:mdt00_002.0 Request processed in 78us (267us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.396405:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23291 00000100:00100000:1.0:1787826277.396575:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112768:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826277.396629:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112896:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826277.396659:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.396740:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826277.396802:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346112896 00000100:00100000:3.0:1787826277.396806:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346112896 00000100:00100000:3.0:1787826277.396869:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23292 00000100:00100000:3.0:1787826277.396878:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346112896:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826277.396984:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346112896:12345-0@lo:1000:mdt00_002.0 Request processed in 107us (323us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826277.396990:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23292 00000100:00100000:2.0:1787826277.397051:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346112896:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1787826277.397135:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.397145:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.397155:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.397169:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f540a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346113024:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787826277.397194:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.397207:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.397856:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346113024 00000100:00100000:0.0:1787826277.397864:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346113024 00000100:00100000:2.0:1787826277.398040:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23293 00000100:00100000:2.0:1787826277.398052:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346113024:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787826277.398065:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826277.398084:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f496ce lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f496c7 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.398142:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f496ce lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f496c7 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.398325:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f496ce lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496c7 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.398333:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f7952ca9900) 00010000:00010000:2.0:1787826277.398335:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826277.398338:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826277.398571:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6529:x1874663346113024:12345-0@lo:101:mdt00_002.0 Request processed in 521us (1375us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.398578:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23293 00000100:00100000:1.0:1787826277.398693:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f540a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346113024:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826277.398715:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.398727:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.398760:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.398768:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.398777:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787826277.399143:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].219, 64 off22208 00000004:00080000:1.0:1787826277.399155:0:6529:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8089611048769ns, next at 8090611046105ns, rc = 0 00000040:00080000:1.0:1787826277.399188:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1175, 384 off1068240 00000040:00080000:1.0:1787826277.399241:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1494, 464 off1507792 00000004:00080000:1.0:1787826277.399246:0:6529:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81f00 with version 6783 00000004:00080000:3.0:1787826277.399359:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6783 rpc_version 6783 00000020:00100000:3.0:1787826277.399386:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826277.399394:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9eb800 x1874663346113280/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1568/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826277.399415:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346113280:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826277.399577:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.399611:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.399669:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346113280 00000100:00100000:1.0:1787826277.399675:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346113280 00000100:00100000:1.0:1787826277.399716:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23294 00000100:00100000:1.0:1787826277.399726:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346113280:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826277.400309:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346113280:12345-0@lo:1000:osp_up0-1.0 Request processed in 583us (731us total) trans 8589957314 rc 0/0 00000100:00100000:1.0:1787826277.400322:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23294 00000100:00100000:3.0:1787826277.400530:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346113280:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826277.400725:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400740:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400749:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400756:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400762:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400767:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400789:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826277.400810:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400816:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.400819:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826277.400822:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.400825:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400828:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826277.400831:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826277.400833:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.400834:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.400841:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400845:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.400853:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f7845cb1a00) 00010000:00010000:0.0:1787826277.400855:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826277.400858:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f7845cb1a00) 00010000:00010000:0.0:1787826277.400859:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400860:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.400864:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.400866:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f496dc lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e41:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400869:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f496d5 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.400877:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.400879:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.400881:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826277.400894:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400902:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400913:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400933:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826277.400945:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18c:0x0] 00010000:00010000:1.0:1787826277.400951:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400959:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400964:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.400995:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787826277.400999:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18c:0x0] 00010000:00010000:1.0:1787826277.401004:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.401011:0:6529:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.401174:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826277.401179:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401181:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826277.401186:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.401187:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826277.401191:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401198:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf200 already unused, calling callback (ffffffffc1986ee0) 00000004:00010000:3.0:1787826277.401200:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401202:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826277.401210:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401236:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.401236:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401243:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826277.401252:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826277.401265:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 1/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.401269:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401273:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f496ab lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496b2 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.401276:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f496c7 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496ce expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401287:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.401288:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826277.401289:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.401290:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826277.401290:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826277.401291:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826277.401304:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346113536:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826277.401341:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.401391:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826277.401399:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346113408:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826277.401432:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.401496:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223612:x1874663332523904:12345-192.168.203.56@tcp:36:rm.0 Request processed in 18958us (20159us total) trans 4294985331 rc 0/0 00000100:00100000:1.0:1787826277.401507:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68223 00000100:00100000:0.0:1787826277.401653:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346113536 00000100:00100000:0.0:1787826277.401660:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346113536 00000100:00100000:0.0:1787826277.401711:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346113408 00000100:00100000:0.0:1787826277.401714:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346113408 00000100:00100000:0.0:1787826277.401747:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10995 00000100:00100000:0.0:1787826277.401756:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346113536:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826277.401766:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826277.401783:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f496b2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f496ab expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401795:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:0.0:1787826277.401803:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826277.401851:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346113536:12345-0@lo:103:ldlm_bl.0 Request processed in 86us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.401858:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10995 00000100:00100000:0.0:1787826277.401874:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10996 00000100:00100000:0.0:1787826277.401880:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346113408:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826277.401896:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826277.401912:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f496ce lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e41:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f496c7 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.401921:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e41:0x0].0x0 (ffff9f7952ca9900) 00000100:00100000:3.0:1787826277.401923:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346113536:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826277.401927:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826277.401950:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346113408:12345-0@lo:103:ldlm_bl.0 Request processed in 70us (517us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.401953:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346113408:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826277.403092:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10996 00000100:00100000:2.0:1787826277.404031:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.404312:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332524416 00000100:00100000:1.0:1787826277.404322:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332524416 00000100:00100000:1.0:1787826277.404756:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9302 00000100:00100000:1.0:1787826277.404769:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223612:x1874663332524416:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:1.0:1787826277.405018:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:223612:x1874663332524416:12345-192.168.203.56@tcp:35:rm.0 Request processed in 252us (992us total) trans 4294985332 rc 0/0 00000100:00100000:1.0:1787826277.405026:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9302 00000100:00100000:2.0:1787826277.412324:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.412511:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332524160 00000100:00100000:3.0:1787826277.412521:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332524160 00000100:00100000:3.0:1787826277.412634:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 10997 00000100:00100000:3.0:1787826277.412645:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c83c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332524160:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826277.412661:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826277.412688:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49507 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ec2 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.412706:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:3.0:1787826277.412712:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826277.412772:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c83c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332524160:12345-192.168.203.56@tcp:103:rm.0 Request processed in 129us (457us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.412780:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 10997 00000100:00100000:3.0:1787826277.419046:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.421610:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332524672 00000100:00100000:2.0:1787826277.421625:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332524672 00000100:00100000:2.0:1787826277.421806:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68224 00000100:00100000:2.0:1787826277.421817:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:223612:x1874663332524672:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:2.0:1787826277.421895:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826277.421924:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f49673 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e40:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f39 expref: 14 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.421951:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4963b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f1d expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.421964:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f49515 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ec9 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.421974:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:2.0:1787826277.421978:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826277.422024:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.422033:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.422044:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.422067:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346113664:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1787826277.422166:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.422209:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.422470:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346113664 00000100:00100000:1.0:1787826277.422477:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346113664 00000100:00100000:1.0:1787826277.422533:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23295 00000100:00100000:1.0:1787826277.422544:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346113664:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826277.422560:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826277.422574:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f496e3 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.422605:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.422614:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.422623:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.422647:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.422661:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 7 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.422672:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346113792:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826277.422759:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.422769:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 7 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.422778:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346113920:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826277.422801:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.430948:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346113792:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826277.430967:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346113920:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826277.431020:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.431034:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826277.431038:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.431043:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.431055:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826277.431183:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:17813:x1874663346113664:12345-0@lo:101:mdt00_005.0 Request processed in 8641us (9015us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.431192:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23295 00000100:00100000:2.0:1787826277.431431:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346113664:0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826277.431465:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.431481:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.434956:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.435013:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.435840:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332524800 00000100:00100000:1.0:1787826277.435858:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332524800 00010000:00010000:1.0:1787826277.435968:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 8 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.436021:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332524928 00000100:00100000:1.0:1787826277.436025:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332524928 00010000:00010000:1.0:1787826277.436041:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 8 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.436065:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21846 00000100:00100000:1.0:1787826277.436077:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332524800:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826277.436097:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826277.436127:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 8 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436144:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 8 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436181:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49554 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ee5 expref: 8 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436203:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826277.436210:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436222:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826277.436298:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332524800:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 223us (1358us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.436307:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21846 00000100:00100000:1.0:1787826277.436328:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21847 00000100:00100000:1.0:1787826277.436335:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332524928:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826277.436345:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826277.436359:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 8 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436726:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 8 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436745:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f494eb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ebb expref: 8 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436763:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826277.436768:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436778:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436789:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.436829:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.436846:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346114048:0@lo:105: 00000100:00100000:1.0:1787826277.436915:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.436974:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.437035:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346114048 00000100:00100000:0.0:1787826277.437042:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346114048 00000100:00100000:0.0:1787826277.437086:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7809 00000100:00100000:0.0:1787826277.437094:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346114048:12345-0@lo:105: 00010000:00010000:0.0:1787826277.437124:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437135:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437158:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437166:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437174:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437183:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437210:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490efc00) 00000100:00100000:0.0:1787826277.437253:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346114048:12345-0@lo:105: Request processed in 160us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.437260:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7809 00010000:00010000:0.0:1787826277.437289:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437300:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826277.437335:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826277.437607:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437617:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.437638:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826277.437655:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346114176:0@lo:101:mdt00_005.0 00000100:00100000:0.0:1787826277.437687:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826277.437706:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.437749:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346114048:0@lo:105: 00010000:00010000:0.0:1787826277.437763:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826277.437793:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332524928:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1458us (2780us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.437802:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21847 00000100:00080000:3.0:1787826277.437956:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826277.437976:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346114176 00000100:00100000:3.0:1787826277.437982:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346114176 00000100:00100000:3.0:1787826277.438028:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23296 00000100:00100000:3.0:1787826277.438038:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:17813:x1874663346114176:12345-0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787826277.438059:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826277.438077:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f496f8 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f496f1 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.438100:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f496f8 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f496f1 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.438158:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f496f8 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496f1 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.438167:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79460a2e00) 00010000:00010000:3.0:1787826277.438170:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826277.438173:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826277.438210:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346114176:12345-0@lo:101:mdt00_005.0 Request processed in 175us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.438216:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23296 00000100:00100000:0.0:1787826277.439218:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346114176:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826277.439248:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.439263:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826277.439288:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826277.439361:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346114304:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826277.439793:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826277.439828:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.440058:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346114304 00000100:00100000:1.0:1787826277.440068:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346114304 00000100:00100000:1.0:1787826277.440204:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23297 00000100:00100000:1.0:1787826277.440216:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346114304:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826277.440325:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346114304:12345-0@lo:1000:mdt00_005.0 Request processed in 111us (534us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.440334:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23297 00000100:00100000:3.0:1787826277.440614:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346114304:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787826277.440777:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346114432:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787826277.440814:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.440841:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826277.440911:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346114432 00000100:00100000:2.0:1787826277.440916:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346114432 00000100:00100000:2.0:1787826277.440955:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23298 00000100:00100000:2.0:1787826277.440963:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346114432:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787826277.441090:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346114432:12345-0@lo:1000:mdt00_005.0 Request processed in 127us (273us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826277.441096:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23298 00000100:00100000:1.0:1787826277.441259:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346114432:0@lo:1000:mdt00_005.0 00010000:00010000:1.0:1787826277.441333:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.441347:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.441756:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.441776:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346114560:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1787826277.441820:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.441854:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826277.446919:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346114560 00000100:00100000:2.0:1787826277.446951:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346114560 00000100:00100000:2.0:1787826277.447055:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23299 00000100:00100000:2.0:1787826277.447069:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346114560:12345-0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826277.447099:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826277.447291:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f496ff expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.447334:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.447345:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f496ff expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.447571:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.447590:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 6 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.447609:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346114688:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826277.447818:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826277.452770:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.453269:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346114688:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826277.453322:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496ff expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.453338:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:2.0:1787826277.453342:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826277.453346:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496ff expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.453364:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826277.453459:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:17813:x1874663346114560:12345-0@lo:101:mdt00_005.0 Request processed in 6392us (11637us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.453470:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23299 00000100:00100000:0.0:1787826277.453642:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332525056 00000100:00100000:0.0:1787826277.453653:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332525056 00010000:00010000:0.0:1787826277.453701:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 7 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826277.453746:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21848 00000100:00100000:0.0:1787826277.453758:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332525056:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826277.453778:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826277.453786:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 7 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.453800:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 7 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.453828:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49570 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eec expref: 7 pid: 17813 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.453848:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:0.0:1787826277.453853:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496ff expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.453865:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f496ff expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.453876:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f496ff expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.453929:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496ff expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826277.453946:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346114816:0@lo:105: 00000100:00100000:0.0:1787826277.453998:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826277.454047:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.454441:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346114560:0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826277.454463:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.454475:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.454740:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346114816 00000100:00100000:2.0:1787826277.454748:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346114816 00000100:00100000:2.0:1787826277.454800:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7810 00000100:00100000:2.0:1787826277.454810:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346114816:12345-0@lo:105: 00010000:00010000:2.0:1787826277.454822:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.454834:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.454845:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.454853:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.454862:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.454872:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.454916:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf800) 00000100:00100000:2.0:1787826277.454985:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346114816:12345-0@lo:105: Request processed in 176us (987us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.454994:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7810 00010000:00010000:1.0:1787826277.455188:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455199:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455246:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455257:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455271:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455293:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455305:0:17813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455313:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455341:0:17813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.455364:0:17813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 11 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.455382:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001_UUID:17813:1874663346114944:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826277.455608:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826277.456301:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346114816:0@lo:105: 00010000:00010000:3.0:1787826277.456325:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826277.456546:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332525056:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2787us (3788us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.456558:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21848 00000100:00100000:3.0:1787826277.461688:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.461955:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001_UUID:17813:1874663346114944:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826277.461997:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826277.462091:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332525184 00000100:00100000:0.0:1787826277.462101:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332525184 00010000:00010000:0.0:1787826277.462204:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 12 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826277.462258:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21849 00000100:00100000:0.0:1787826277.462270:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332525184:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826277.462289:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826277.462297:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 12 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.462311:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 12 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.462341:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49649 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f24 expref: 12 pid: 6529 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.462362:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:0.0:1787826277.462368:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.462390:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.462399:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.462428:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826277.462504:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332525184:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 235us (830us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.462512:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21849 00010000:00010000:1.0:1787826277.463136:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.463155:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.463216:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.463225:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.463237:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.463260:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346115072:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1787826277.463319:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.463362:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826277.463482:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346115072 00000100:00100000:2.0:1787826277.463491:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346115072 00000100:00100000:2.0:1787826277.463539:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23300 00000100:00100000:2.0:1787826277.463549:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:17813:x1874663346115072:12345-0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826277.463570:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826277.463590:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49722 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4971b expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.463613:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49722 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4971b expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.463637:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49722 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4971b expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.463646:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f7952ca9f00) 00010000:00010000:2.0:1787826277.463649:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826277.463652:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826277.463702:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:17813:x1874663346115072:12345-0@lo:101:mdt00_005.0 Request processed in 155us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.463709:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23300 00000100:00100000:1.0:1787826277.463911:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346115072:0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826277.463930:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.463942:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826277.466022:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91df80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346115200:0@lo:403:mdt00_005.0 00000100:00100000:0.0:1787826277.466485:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826277.466533:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.466868:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346115200 00000100:00100000:1.0:1787826277.466881:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346115200 00000100:00100000:1.0:1787826277.466969:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23301 00000100:00100000:1.0:1787826277.466982:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8cc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:17813:x1874663346115200:12345-0@lo:403:mdt00_005.0 00000100:00100000:0.0:1787826277.472395:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8cc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:17813:x1874663346115200:12345-0@lo:403:mdt00_005.0 Request processed in 5401us (5896us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.472422:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23301 00000100:00100000:3.0:1787826277.473075:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91df80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346115200:0@lo:403:mdt00_005.0 00000040:00080000:3.0:1787826277.476066:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1176, 952 off1069192 00000040:00080000:3.0:1787826277.476584:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1495, 1032 off1508824 00000004:00080000:3.0:1787826277.476593:0:17813:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78500aa780 with version 6784 00000004:00080000:1.0:1787826277.476775:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6784 rpc_version 6784 00000020:00100000:1.0:1787826277.476829:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826277.476845:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977c6ca80 x1874663346115456/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826277.476896:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346115456:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826277.477541:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.477667:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826277.477731:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346115456 00000100:00100000:2.0:1787826277.477741:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346115456 00000100:00100000:3.0:1787826277.478580:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23302 00000100:00100000:3.0:1787826277.478606:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346115456:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826277.480506:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346115456:12345-0@lo:1000:osp_up0-1.0 Request processed in 1895us (2957us total) trans 8589957315 rc 0/0 00000100:00100000:3.0:1787826277.480524:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23302 00000100:00100000:1.0:1787826277.481363:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346115456:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787826277.481513:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e40:0x0]: returning size 4096 00010000:00010000:3.0:1787826277.481548:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481563:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481572:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481579:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826277.481584:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481593:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481615:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18e:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:3.0:1787826277.481618:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826277.481692:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481701:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49714 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481731:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481740:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481747:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481831:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826277.481844:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18c:0x0] 00010000:00010000:3.0:1787826277.481855:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481862:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481868:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481874:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826277.481878:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481894:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481903:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:3.0:1787826277.481906:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826277.481909:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481915:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4970d lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481925:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481932:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.481941:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.483989:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826277.484009:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484037:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826277.484046:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826277.484058:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484143:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484161:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826277.484216:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 1/0,0 mode: --/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484230:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecec00/0xe21309aa81f4971b lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49722 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484259:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826277.484264:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484271:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826277.484275:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826277.484282:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484302:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484309:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826277.484331:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484342:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.484344:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.484346:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826277.484384:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346115712:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826277.484476:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826277.484554:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346115712 00000100:00100000:0.0:1787826277.484563:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346115712 00000100:00100000:0.0:1787826277.484620:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10998 00000100:00100000:0.0:1787826277.484642:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3654:x1874663346115712:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826277.484657:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826277.484677:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49706 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f496ff expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.484693:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:0.0:1787826277.484704:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826277.484737:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+14:3654:x1874663346115712:12345-0@lo:103:ldlm_bl.0 Request processed in 97us (261us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.484744:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10998 00000100:00100000:0.0:1787826277.484786:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346115712:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826277.485132:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346115584:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826277.485178:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.485218:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346115584 00000100:00100000:2.0:1787826277.485222:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346115584 00000100:00100000:2.0:1787826277.485257:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10999 00000100:00100000:2.0:1787826277.485265:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+14:3653:x1874663346115584:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826277.485273:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826277.485286:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49722 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4971b expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.485300:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x0 (ffff9f7952ca9f00) 00010000:00010000:2.0:1787826277.485309:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826277.485333:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346115584:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (155us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.485340:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10999 00000100:00100000:2.0:1787826277.485374:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346115584:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826277.486421:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1787826277.486437:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f496ff lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e40:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49706 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826277.486482:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826277.486513:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.486523:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.486531:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.486706:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826277.486714:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.486724:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efc00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826277.486730:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826277.486740:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.486788:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.486794:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826277.486859:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346115840:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826277.486938:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826277.487000:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.487012:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.487014:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.487016:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826277.487072:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346115840 00000100:00100000:2.0:1787826277.487080:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346115840 00000100:00100000:2.0:1787826277.487167:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11000 00000100:00100000:2.0:1787826277.487178:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c80c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346115840:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826277.487192:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826277.487211:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f496ea lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f496e3 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.487226:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826277.487231:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826277.487388:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1787826277.487394:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efc00/0xe21309aa81f496e3 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496ea expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826277.487418:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826277.487424:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.487433:0:17813:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.487524:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223612:x1874663332524672:12345-192.168.203.56@tcp:36:rm.0 Request processed in 65706us (68486us total) trans 4294985333 rc 0/0 00000100:00100000:3.0:1787826277.487534:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68224 00010000:00010000:0.0:1787826277.487764:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826277.487861:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c80c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346115840:12345-0@lo:103:ldlm_bl.0 Request processed in 683us (921us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.487870:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11000 00000100:00100000:1.0:1787826277.487986:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346115840:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826277.548334:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826277.558692:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826277.558822:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332525312 00000100:00100000:0.0:1787826277.558835:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332525312 00000100:00100000:0.0:1787826277.558926:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68225 00000100:00100000:0.0:1787826277.558941:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223620:x1874663332525312:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:0.0:1787826277.558998:0:6527:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345651712 00000100:00100000:0.0:1787826277.559188:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223620:x1874663332525312:12345-192.168.203.56@tcp:41:stat.0 Request processed in 249us (579us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.559198:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68225 00000100:00100000:1.0:1787826277.606594:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.606767:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332525440 00000100:00100000:3.0:1787826277.606782:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332525440 00000100:00100000:3.0:1787826277.606860:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68226 00000100:00100000:3.0:1787826277.606874:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223624:x1874663332525440:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826277.606914:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826277.606944:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49729 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f4e expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.607045:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.607064:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.607077:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826277.607200:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826277.607205:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826277.607231:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.607260:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49729 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.607281:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.607291:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826277.607295:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826277.607299:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826277.607386:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223624:x1874663332525440:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 515us (810us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.607392:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68226 00000100:00100000:2.0:1787826277.617656:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.617780:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332525568 00000100:00100000:1.0:1787826277.617795:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332525568 00000100:00100000:1.0:1787826277.617895:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68227 00000100:00100000:1.0:1787826277.617910:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223624:x1874663332525568:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826277.617940:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826277.617967:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49737 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f55 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.618062:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.618082:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.618095:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.618165:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.618178:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f496f8 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f496f1 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.618186:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.618229:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f496f8 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f496f1 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.618250:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346115968:0@lo:104: 00000100:00100000:1.0:1787826277.618344:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.618405:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826277.618461:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826277.618476:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346115968 00000100:00100000:3.0:1787826277.618481:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346115968 00000100:00100000:3.0:1787826277.618531:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7811 00000100:00100000:3.0:1787826277.618541:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7855a37450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346115968:12345-0@lo:104: 00010000:00010000:3.0:1787826277.618555:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.618630:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826277.618637:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7855a37450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346115968:12345-0@lo:104: Request processed in 98us (292us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.618645:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7811 00010000:00010000:0.0:1787826277.618803:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826277.618813:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.618821:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826277.618828:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826277.618837:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.618869:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.618891:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826277.618921:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.618930:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f496f1 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f496f8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.618952:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:0.0:1787826277.618953:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.618955:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826277.618957:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826277.618967:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346116096:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826277.619008:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.619071:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346115968:0@lo:104: 00010000:00010000:1.0:1787826277.619094:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826277.619271:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346116096 00000100:00100000:0.0:1787826277.619278:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346116096 00010000:00010000:0.0:1787826277.619304:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f496f8 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f496f1 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826277.619338:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21850 00000100:00100000:0.0:1787826277.619348:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346116096:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826277.619362:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787826277.622518:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957315 is committed 00010000:00010000:0.0:1787826277.624543:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f496f8 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f496f1 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.624577:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79460a2e00) 00010000:00010000:0.0:1787826277.624584:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.624596:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.624611:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.624787:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826277.624922:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346116096:12345-0@lo:103:ldlm_bl.0 Request processed in 5572us (5909us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.624935:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21850 00000100:00100000:2.0:1787826277.625285:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346116096:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826277.627344:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627473:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627768:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627778:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627786:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627796:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627802:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826277.627807:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627815:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627837:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826277.627841:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.627845:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627852:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4973e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627871:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627876:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627893:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627900:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826277.627903:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627914:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627922:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79460a2e00) 00010000:00010000:1.0:1787826277.627924:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.627928:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.627934:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49745 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.628002:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49737 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f55 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.628014:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826277.628016:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.628020:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49737 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f55 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.628030:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826277.629497:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223624:x1874663332525568:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 11573us (11840us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826277.629513:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68227 00000100:00100000:2.0:1787826277.635231:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.635581:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332525696 00000100:00100000:1.0:1787826277.635595:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332525696 00000100:00100000:1.0:1787826277.635680:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68228 00000100:00100000:1.0:1787826277.635693:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223624:x1874663332525696:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826277.635726:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826277.635755:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4974c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f5c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.635843:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.635859:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.635872:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826277.635950:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826277.635954:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826277.635979:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.636010:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4974c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.636031:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.636041:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826277.636044:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826277.636049:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826277.636500:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223624:x1874663332525696:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 805us (1278us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.636509:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68228 00000100:00100000:2.0:1787826277.642668:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826277.642910:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332525952 00000100:00100000:0.0:1787826277.642924:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332525952 00000100:00100000:0.0:1787826277.643023:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68229 00000100:00100000:0.0:1787826277.643036:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223624:x1874663332525952:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:0.0:1787826277.643229:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346116224:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826277.643296:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826277.643329:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.643554:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346116224 00000100:00100000:1.0:1787826277.643561:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346116224 00000100:00100000:1.0:1787826277.643603:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23303 00000100:00100000:1.0:1787826277.643612:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346116224:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826277.643717:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346116224:12345-0@lo:1000:mdt00_006.0 Request processed in 106us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.643725:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23303 00000100:00100000:0.0:1787826277.643874:0:24702:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957315 gen 2 00000004:00080000:0.0:1787826277.643892:0:24702:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957313 committed transno 8589957315 00000004:00080000:0.0:1787826277.643922:0:24702:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957314 committed transno 8589957315 00000004:00080000:0.0:1787826277.643940:0:24702:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957315 committed transno 8589957315 00000100:00100000:0.0:1787826277.643956:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346116224:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826277.644009:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346116352:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826277.644036:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826277.644063:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.644079:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346116352 00000100:00100000:0.0:1787826277.644084:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346116352 00000100:00100000:0.0:1787826277.644542:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23304 00000100:00100000:0.0:1787826277.644556:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346116352:12345-0@lo:1000:mdt00_006.0 00000100:00080000:3.0:1787826277.644827:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826277.644874:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346116352:12345-0@lo:1000:mdt00_006.0 Request processed in 321us (836us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826277.644893:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23304 00000100:00100000:2.0:1787826277.645540:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346116352:0@lo:1000:mdt00_006.0 00010000:00010000:2.0:1787826277.645639:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.645652:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.645668:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.645688:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346116480:0@lo:101:mdt00_006.0 00000100:00100000:2.0:1787826277.645736:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.645925:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f253c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.648324:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346116480 00000100:00100000:1.0:1787826277.648346:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346116480 00000100:00100000:1.0:1787826277.648636:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23305 00000100:00100000:1.0:1787826277.648650:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346116480:12345-0@lo:101:mdt00_006.0 00010000:00010000:1.0:1787826277.648677:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826277.648703:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4975a expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.648741:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.648750:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.648761:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.648801:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.648816:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 7 pid: 35094 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.648831:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346116608:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826277.648969:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.648979:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 6 pid: 35080 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.648988:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6e680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346116736:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826277.649012:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e813c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.658961:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346116608:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826277.658983:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6e680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346116736:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826277.659031:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.659047:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826277.659050:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.659056:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.659068:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826277.659199:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:24702:x1874663346116480:12345-0@lo:101:mdt00_006.0 Request processed in 10551us (13461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.659209:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23305 00000100:00100000:2.0:1787826277.659518:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346116480:0@lo:101:mdt00_006.0 00010000:00010000:2.0:1787826277.659541:0:24702:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.659554:0:24702:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.659894:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.659943:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.660784:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332526080 00000100:00100000:3.0:1787826277.660797:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332526080 00010000:00010000:3.0:1787826277.660845:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 8 pid: 35094 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.660906:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332526208 00000100:00100000:3.0:1787826277.660910:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332526208 00010000:00010000:3.0:1787826277.660926:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 7 pid: 35080 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.660962:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21851 00000100:00100000:3.0:1787826277.660974:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332526080:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826277.660991:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826277.661000:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 8 pid: 35094 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.661015:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 8 pid: 35094 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.661041:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49753 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f5c expref: 8 pid: 35094 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.661063:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826277.661070:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.661080:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826277.661192:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332526080:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 220us (1310us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.661200:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21851 00000100:00100000:3.0:1787826277.661218:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21852 00000100:00100000:3.0:1787826277.661225:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332526208:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826277.661233:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826277.661237:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 7 pid: 35080 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.661247:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 7 pid: 35080 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.663091:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49730 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f4e expref: 7 pid: 35080 timeout: 8189 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.663130:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826277.663137:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.663150:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.663159:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.663214:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.663232:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346116864:0@lo:105: 00000100:00100000:3.0:1787826277.663383:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.663609:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.663966:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346116864 00000100:00100000:0.0:1787826277.663977:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346116864 00000100:00100000:0.0:1787826277.664081:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7812 00000100:00100000:0.0:1787826277.664094:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346116864:12345-0@lo:105: 00010000:00010000:0.0:1787826277.664134:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.664148:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.664159:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.664168:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.664176:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.664185:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.664214:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eea00) 00000100:00100000:0.0:1787826277.664295:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346116864:12345-0@lo:105: Request processed in 203us (912us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.664305:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7812 00000100:00100000:1.0:1787826277.664416:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346116864:0@lo:105: 00010000:00010000:1.0:1787826277.664441:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826277.664510:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332526208:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3282us (4564us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.664519:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21852 00010000:00010000:2.0:1787826277.665045:0:24702:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.665059:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826277.665093:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826277.665160:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.665170:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.665182:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.665205:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346116992:0@lo:101:mdt00_006.0 00000100:00100000:2.0:1787826277.665266:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.665306:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f253c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.665521:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346116992 00000100:00100000:1.0:1787826277.665530:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346116992 00000100:00100000:1.0:1787826277.665582:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23306 00000100:00100000:1.0:1787826277.665592:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:24702:x1874663346116992:12345-0@lo:101:mdt00_006.0 00010000:00010000:1.0:1787826277.665614:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826277.665641:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4976f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49768 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.665670:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4976f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49768 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.665684:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4976f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49768 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.665694:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1700) 00010000:00010000:1.0:1787826277.665697:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.665700:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826277.665746:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346116992:12345-0@lo:101:mdt00_006.0 Request processed in 155us (479us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.665753:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23306 00000100:00100000:3.0:1787826277.666360:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346116992:0@lo:101:mdt00_006.0 00010000:00010000:3.0:1787826277.666384:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.666397:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826277.666434:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826277.666497:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117120:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.666542:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.666575:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483dbcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.666819:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346117120 00000100:00100000:0.0:1787826277.666827:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346117120 00000100:00100000:0.0:1787826277.666870:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23307 00000100:00100000:0.0:1787826277.666893:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117120:12345-0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826277.666983:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117120:12345-0@lo:1000:mdt00_006.0 Request processed in 104us (439us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.666992:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23307 00000100:00100000:3.0:1787826277.668164:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117120:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.668274:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117248:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.668333:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.668375:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f79438bae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.668586:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346117248 00000100:00100000:1.0:1787826277.668595:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346117248 00000100:00100000:1.0:1787826277.668663:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23308 00000100:00100000:1.0:1787826277.668675:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117248:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826277.668802:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117248:12345-0@lo:1000:mdt00_006.0 Request processed in 129us (468us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826277.668812:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23308 00000100:00100000:3.0:1787826277.669424:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117248:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.669554:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117376:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.669595:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.669614:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f79438bae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.670155:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346117376 00000100:00100000:0.0:1787826277.670164:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346117376 00000100:00100000:0.0:1787826277.670216:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23309 00000100:00100000:0.0:1787826277.670226:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117376:12345-0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826277.670345:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117376:12345-0@lo:1000:mdt00_006.0 Request processed in 119us (749us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826277.670643:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23309 00000100:00100000:3.0:1787826277.670916:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117376:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.671001:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117504:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.671038:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.671066:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f79438bae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.671137:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346117504 00000100:00100000:1.0:1787826277.671144:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346117504 00000100:00100000:1.0:1787826277.671195:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23310 00000100:00100000:1.0:1787826277.671205:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117504:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826277.671290:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117504:12345-0@lo:1000:mdt00_006.0 Request processed in 87us (252us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.671299:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23310 00000100:00100000:3.0:1787826277.671734:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117504:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.671789:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117632:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.671815:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.671836:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f79438bae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.672008:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346117632 00000100:00100000:1.0:1787826277.672013:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346117632 00000100:00100000:1.0:1787826277.672049:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23311 00000100:00100000:1.0:1787826277.672057:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117632:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826277.672150:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117632:12345-0@lo:1000:mdt00_006.0 Request processed in 93us (334us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.672257:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23311 00000100:00100000:3.0:1787826277.672471:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117632:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.672578:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117760:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826277.672609:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.672716:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f79438bae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.672721:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346117760 00000100:00100000:1.0:1787826277.672726:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346117760 00000100:00100000:1.0:1787826277.672803:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23312 00000100:00100000:1.0:1787826277.672812:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117760:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826277.672880:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346117760:12345-0@lo:1000:mdt00_006.0 Request processed in 69us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.672896:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23312 00000100:00100000:3.0:1787826277.672951:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346117760:0@lo:1000:mdt00_006.0 00000040:00080000:3.0:1787826277.674756:0:24702:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1177, 2168 off1071360 00000040:00080000:3.0:1787826277.674857:0:24702:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1496, 2368 off1511192 00000004:00080000:3.0:1787826277.674864:0:24702:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f79438bad80 with version 6785 00000004:00080000:1.0:1787826277.674962:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6785 rpc_version 6785 00000020:00100000:1.0:1787826277.675012:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826277.675027:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977c6e680 x1874663346118016/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826277.675055:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6e680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346118016:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826277.675223:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.675313:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e813c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.675578:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346118016 00000100:00100000:0.0:1787826277.675589:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346118016 00000100:00100000:0.0:1787826277.675762:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23313 00000100:00100000:0.0:1787826277.675774:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346118016:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826277.677068:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346118016:12345-0@lo:1000:osp_up0-1.0 Request processed in 1290us (1838us total) trans 8589957316 rc 0/0 00000100:00100000:0.0:1787826277.677081:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23313 00000100:00100000:3.0:1787826277.677425:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6e680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346118016:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826277.677768:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.677790:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.677806:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.677835:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.677843:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.677850:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.677904:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.677912:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.677925:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.677949:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346118144:0@lo:101:mdt00_006.0 00000100:00100000:2.0:1787826277.678004:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.678667:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.678793:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346118144 00000100:00100000:1.0:1787826277.678804:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346118144 00000100:00100000:1.0:1787826277.679056:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23314 00000100:00100000:1.0:1787826277.679067:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:24702:x1874663346118144:12345-0@lo:101:mdt00_006.0 00010000:00010000:1.0:1787826277.679093:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826277.679140:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4978b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49784 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.679176:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4978b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49784 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.679191:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4978b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49784 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.679201:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:1.0:1787826277.679205:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.679208:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826277.679481:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346118144:12345-0@lo:101:mdt00_006.0 Request processed in 414us (1476us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.679563:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23314 00000100:00100000:2.0:1787826277.679700:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346118144:0@lo:101:mdt00_006.0 00010000:00010000:2.0:1787826277.679721:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679735:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826277.679763:0:24702:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007a6a6d97 reply state 000000007719eb9e transno 4294985334 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679780:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679788:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:2.0:1787826277.679791:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826277.679795:0:24702:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826277.679802:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679811:0:24702:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826277.679825:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18d:0x0] 00000004:00010000:2.0:1787826277.679834:0:24702:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007a6a6d97 reply state 000000007719eb9e transno 4294985334 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679844:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679850:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:2.0:1787826277.679853:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826277.679855:0:24702:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787826277.679861:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e42:0x0]: returning size 4096 00000004:00080000:2.0:1787826277.679872:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826277.679899:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679908:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679920:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679940:0:24702:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826277.679945:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826277.679950:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.679957:0:24702:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.680062:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223624:x1874663332525952:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 37025us (37407us total) trans 4294985334 rc 0/0 00000100:00100000:2.0:1787826277.680070:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68229 00010000:00010000:1.0:1787826277.680536:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826277.680545:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.680560:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826277.680566:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826277.680575:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.680616:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.680632:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826277.680660:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.680668:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4975a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49761 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.680686:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826277.680688:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826277.680690:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826277.680699:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826277.680717:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346118272:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826277.680775:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.680821:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346118272 00000100:00100000:1.0:1787826277.680829:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346118272 00000100:00100000:1.0:1787826277.680891:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11001 00000100:00100000:1.0:1787826277.680901:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346118272:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826277.680918:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826277.680938:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49761 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4975a expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.680953:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826277.680957:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826277.680988:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346118272:12345-0@lo:103:ldlm_bl.0 Request processed in 89us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.680994:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11001 00000100:00100000:1.0:1787826277.681028:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346118272:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826277.701093:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826277.706833:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332526336 00000100:00100000:0.0:1787826277.706859:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332526336 00000100:00100000:2.0:1787826277.707080:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68230 00000100:00100000:2.0:1787826277.707095:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223613:x1874663332526336:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826277.707156:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826277.707186:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49792 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f63 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.707283:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49799 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.707302:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49799 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.707320:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49799 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826277.707734:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826277.707739:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826277.707772:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49799 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.707814:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f49792 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f63 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.707840:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49799 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f63 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.707851:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826277.707854:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826277.707859:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826277.707971:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223613:x1874663332526336:12345-192.168.203.56@tcp:101:bash.0 Request processed in 878us (6894us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.707980:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68230 00000100:00100000:2.0:1787826277.711978:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.712604:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332526464 00000100:00100000:3.0:1787826277.712618:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332526464 00000100:00100000:3.0:1787826277.712727:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68231 00000100:00100000:3.0:1787826277.712745:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223613:x1874663332526464:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826277.712781:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826277.712820:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f497a0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f6a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.712951:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.712973:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.712989:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.713005:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.713016:0:35093:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4976f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49768 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.713025:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.713082:0:35093:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4976f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49768 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.713101:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000_UUID:35093:1874663346118400:0@lo:104: 00000100:00100000:3.0:1787826277.713231:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826277.713289:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f796cce2f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826277.714853:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346118400 00000100:00100000:2.0:1787826277.714872:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346118400 00000100:00100000:2.0:1787826277.714968:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7813 00000100:00100000:2.0:1787826277.714981:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35093:x1874663346118400:12345-0@lo:104: 00010000:00010000:2.0:1787826277.715001:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.715317:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826277.715326:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35093:x1874663346118400:12345-0@lo:104: Request processed in 347us (2094us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.715334:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7813 00010000:00010000:3.0:1787826277.715985:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826277.715997:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.716009:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826277.716016:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826277.716027:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.716081:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.716095:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826277.716166:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.716174:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f49768 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4976f expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.716202:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826277.716205:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826277.716207:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826277.716333:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346118528:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826277.720542:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000_UUID:35093:1874663346118400:0@lo:104: 00010000:00010000:0.0:1787826277.720590:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.723584:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826277.723921:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346118528 00000100:00100000:1.0:1787826277.723935:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346118528 00010000:00010000:1.0:1787826277.723994:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4976f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49768 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826277.724047:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21853 00000100:00100000:1.0:1787826277.724058:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346118528:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826277.724077:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787826277.729240:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957316 is committed 00010000:00010000:1.0:1787826277.734566:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4976f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49768 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.734601:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1700) 00010000:00010000:1.0:1787826277.734608:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.734685:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.734694:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.734911:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826277.735037:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346118528:12345-0@lo:103:ldlm_bl.0 Request processed in 10977us (11460us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.735051:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21853 00000100:00100000:1.0:1787826277.735202:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346118528:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826277.735452:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.735464:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.735719:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-mdtlov_UUID:35093:1874663346118656:0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826277.735780:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.735857:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826277.735920:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346118656 00000100:00100000:1.0:1787826277.735928:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346118656 00000100:00100000:1.0:1787826277.736029:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23315 00000100:00100000:1.0:1787826277.736040:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:35093:x1874663346118656:12345-0@lo:1000:mdt00_010.0 00000100:00100000:1.0:1787826277.736180:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:35093:x1874663346118656:12345-0@lo:1000:mdt00_010.0 Request processed in 142us (399us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.736188:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23315 00000100:00100000:3.0:1787826277.736266:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-mdtlov_UUID:35093:1874663346118656:0@lo:1000:mdt00_010.0 00010000:00010000:3.0:1787826277.736476:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f497b5 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736501:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f497b5 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736514:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f497b5 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826277.736554:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f497b5 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736572:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736579:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736589:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736596:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736603:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826277.736608:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736615:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736629:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826277.736633:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826277.736637:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736644:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497a7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736664:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736670:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736677:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736682:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826277.736686:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736692:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736699:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1700) 00010000:00010000:3.0:1787826277.736702:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826277.736705:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736711:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f497ae lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736753:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f497a0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f6a expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736769:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f497b5 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f6a expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.736777:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:3.0:1787826277.736779:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826277.736782:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826277.736857:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223613:x1874663332526464:12345-192.168.203.56@tcp:101:bash.0 Request processed in 24118us (24893us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.736866:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68231 00000100:00100000:3.0:1787826277.745059:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826277.745202:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332526592 00000100:00100000:0.0:1787826277.745216:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332526592 00000100:00100000:0.0:1787826277.745293:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68232 00000100:00100000:0.0:1787826277.745306:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223613:x1874663332526592:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826277.745341:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826277.745512:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f497bc lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f71 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.745597:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f497c3 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.745616:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f497c3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.745705:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f497c3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826277.745850:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e42:0x0]: returning size 4096 00000004:00200000:0.0:1787826277.745856:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826277.745917:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f497bc lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f71 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.745941:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f497c3 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f71 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.745958:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:0.0:1787826277.745961:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826277.745966:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826277.746041:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223613:x1874663332526592:12345-192.168.203.56@tcp:101:bash.0 Request processed in 737us (997us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.746049:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68232 00000100:00100000:0.0:1787826277.750820:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.750941:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332526720 00000100:00100000:3.0:1787826277.750955:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332526720 00000100:00100000:3.0:1787826277.751073:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68233 00000100:00100000:3.0:1787826277.751086:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223613:x1874663332526720:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826277.751144:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826277.751178:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f497ca lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f78 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.751263:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f497d1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.751280:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f497d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.751294:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f497d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826277.751374:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18f:0x0]: returning size 4096 00000004:00200000:3.0:1787826277.751381:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826277.751560:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f497ca lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f78 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.751583:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f497d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f78 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.751593:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:3.0:1787826277.751597:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826277.751602:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826277.751742:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223613:x1874663332526720:12345-192.168.203.56@tcp:101:bash.0 Request processed in 657us (935us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.751751:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68233 00000100:00100000:2.0:1787826277.755769:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.755857:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332526848 00000100:00100000:3.0:1787826277.755873:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332526848 00000100:00100000:3.0:1787826277.755965:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68234 00000100:00100000:3.0:1787826277.755978:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223613:x1874663332526848:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826277.756011:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826277.756242:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f7f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.756331:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.756362:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4978b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49784 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.756374:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 4/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.756547:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4978b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49784 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.756568:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6528:1874663346118784:0@lo:104: 00000100:00100000:3.0:1787826277.756723:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826277.757878:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346118784 00000100:00100000:0.0:1787826277.757904:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346118784 00000100:00100000:0.0:1787826277.757978:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7814 00000100:00100000:0.0:1787826277.757991:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6528:x1874663346118784:12345-0@lo:104: 00010000:00010000:0.0:1787826277.758011:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.758326:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f796cce2f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826277.758372:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6528:1874663346118784:0@lo:104: 00010000:00010000:2.0:1787826277.758416:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.758520:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826277.758529:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6528:x1874663346118784:12345-0@lo:104: Request processed in 540us (1806us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.758538:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7814 00010000:00010000:1.0:1787826277.758789:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826277.758800:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.758810:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efe00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826277.758817:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826277.758828:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.758872:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.758894:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826277.758959:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346118912:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826277.759023:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826277.759092:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.759101:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49784 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4978b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.759595:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826277.759599:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826277.759602:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826277.759770:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346118912 00000100:00100000:3.0:1787826277.759780:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346118912 00010000:00010000:3.0:1787826277.759819:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4978b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49784 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826277.759899:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21854 00000100:00100000:3.0:1787826277.759912:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346118912:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826277.759927:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826277.760919:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4978b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49784 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.760943:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:3.0:1787826277.760949:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.760960:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.760969:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.761000:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826277.761190:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.761199:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826277.761280:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18d:0x0]: returning size 4096 00000004:00200000:2.0:1787826277.761287:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826277.761346:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f497d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f7f expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.761377:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f7f expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.761385:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:2.0:1787826277.761388:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826277.761391:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:2.0:1787826277.761462:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223613:x1874663332526848:12345-192.168.203.56@tcp:101:bash.0 Request processed in 5486us (5708us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826277.761471:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68234 00000100:00100000:3.0:1787826277.761791:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346118912:12345-0@lo:103:ldlm_bl.0 Request processed in 1880us (2766us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.761800:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21854 00000100:00100000:0.0:1787826277.767792:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346118912:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826277.773087:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.773561:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332527104 00000100:00100000:1.0:1787826277.773575:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332527104 00000100:00100000:1.0:1787826277.774589:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68235 00000100:00100000:0.0:1787826277.776786:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223613:x1874663332527104:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826277.776824:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826277.776829:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826277.776873:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f497df lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f7f expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.776918:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:0.0:1787826277.776934:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826277.776956:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f497e6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f86 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.777187:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.777203:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.777216:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.777232:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.777239:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.777245:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826277.778025:0:35094:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x51d:0x0], index 1: 0 00010000:00010000:0.0:1787826277.779010:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f497fb lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779031:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f497fb lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779049:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f497fb lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779100:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779282:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779290:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779297:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779304:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826277.779309:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779318:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779332:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f794b57af00) 00010000:00010000:0.0:1787826277.779335:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826277.779340:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779347:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f497ed lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826277.779761:0:35094:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fdf35234 reply state 00000000a41869c0 transno 8589957317 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779777:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779785:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f794b57a600) 00010000:00010000:0.0:1787826277.779788:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826277.779791:0:35094:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826277.779849:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f497e6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f86 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779875:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f497fb lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f86 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826277.779896:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:0.0:1787826277.779899:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826277.779902:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826277.780073:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223613:x1874663332527104:12345-192.168.203.56@tcp:101:bash.0 Request processed in 5470us (7001us total) trans 8589957317 rc 0/0 00000100:00100000:0.0:1787826277.780084:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68235 00000100:00100000:2.0:1787826277.784566:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.784789:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332527232 00000100:00100000:1.0:1787826277.784802:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332527232 00000100:00100000:1.0:1787826277.784932:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30780 00000100:00100000:1.0:1787826277.784946:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223613:x1874663332527232:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826277.784971:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826277.784998:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f8d expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826277.785030:0:24218:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x51d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787826277.785059:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=8089s v=(0 0) 00010000:00010000:1.0:1787826277.785064:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826277.785076:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 7 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.785101:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 7 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.785294:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826277.785528:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223613:x1874663332527232:12345-192.168.203.56@tcp:101:bash.0 Request processed in 584us (974us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.785536:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30780 00000100:00100000:2.0:1787826277.789596:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826277.789692:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332527488 00000100:00100000:3.0:1787826277.789707:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332527488 00000100:00100000:3.0:1787826277.789785:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9303 00000100:00100000:3.0:1787826277.789800:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223613:x1874663332527488:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826277.790385:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223613:x1874663332527488:12345-192.168.203.56@tcp:35:bash.0 Request processed in 585us (802us total) trans 8589957318 rc 0/0 00000100:00100000:3.0:1787826277.790396:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9303 00000100:00100000:0.0:1787826277.793933:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.795283:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332527744 00000100:00100000:2.0:1787826277.795301:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332527744 00000100:00100000:2.0:1787826277.795399:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68236 00000100:00100000:2.0:1787826277.795486:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223613:x1874663332527744:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826277.795516:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826277.795520:0:35092:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826277.795557:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f497d1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f78 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.795578:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:2.0:1787826277.795583:0:35092:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826277.795668:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49810 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f94 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.795849:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.795866:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.795877:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.795907:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.795914:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826277.795921:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826277.796102:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346119040:0@lo:1000:mdt00_009.0 00000100:00100000:2.0:1787826277.797920:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826277.797985:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f253c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826277.798161:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346119040 00000100:00100000:0.0:1787826277.798179:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346119040 00000100:00100000:0.0:1787826277.798241:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23316 00000100:00100000:0.0:1787826277.798262:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346119040:12345-0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826277.798366:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346119040:12345-0@lo:1000:mdt00_009.0 Request processed in 115us (449us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826277.798373:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23316 00000100:00100000:1.0:1787826277.798844:0:35092:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957316 gen 2 00000004:00080000:1.0:1787826277.798853:0:35092:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957316 committed transno 8589957316 00000100:00100000:1.0:1787826277.799738:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346119040:0@lo:1000:mdt00_009.0 00000004:00080000:1.0:1787826277.800188:0:35092:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4ca:0x0], index 0: 0 00010000:00010000:1.0:1787826277.800763:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49825 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800785:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49825 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800800:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49825 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800852:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800861:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800867:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800876:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800896:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826277.800901:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800910:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800926:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:1.0:1787826277.800929:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826277.800934:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800941:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f49817 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826277.800959:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000369aa2fb reply state 0000000010f4e32c transno 4294985335 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800972:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.800977:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x4ec94fd6 (ffff9f7952ca9200) 00010000:00010000:1.0:1787826277.800980:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826277.800983:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826277.801029:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49810 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f94 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.801050:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49825 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f94 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.801058:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f7845cb1000) 00010000:00010000:1.0:1787826277.801061:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826277.801063:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826277.801337:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223613:x1874663332527744:12345-192.168.203.56@tcp:101:bash.0 Request processed in 5854us (7416us total) trans 4294985335 rc 0/0 00000100:00100000:1.0:1787826277.801348:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68236 00000100:00100000:2.0:1787826277.806292:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.806720:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332527872 00000100:00100000:1.0:1787826277.806733:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332527872 00000100:00080000:3.0:1787826277.806873:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826277.806874:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30781 00000100:00100000:1.0:1787826277.806900:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223613:x1874663332527872:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826277.806927:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826277.806956:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800f9b expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826277.806992:0:35096:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4ca:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787826277.807023:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ca:0x0].0x0 since=8090s v=(0 0) 00010000:00010000:1.0:1787826277.807029:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ca:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826277.807042:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 8 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.807063:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 8 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.807086:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826277.807541:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223613:x1874663332527872:12345-192.168.203.56@tcp:101:bash.0 Request processed in 643us (1260us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.807550:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30781 00000100:00100000:2.0:1787826277.811952:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.812358:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332528128 00000100:00100000:1.0:1787826277.812503:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332528128 00000100:00100000:1.0:1787826277.812638:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9304 00000100:00100000:1.0:1787826277.812653:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223613:x1874663332528128:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:1.0:1787826277.813102:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:223613:x1874663332528128:12345-192.168.203.56@tcp:35:bash.0 Request processed in 452us (1167us total) trans 4294985336 rc 0/0 00000100:00100000:1.0:1787826277.813162:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9304 00000100:00100000:2.0:1787826277.835259:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.835367:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826277.835580:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826277.835983:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332528256 00000100:00100000:1.0:1787826277.835998:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332528256 00000100:00100000:1.0:1787826277.836808:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332528384 00000100:00100000:1.0:1787826277.836818:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332528384 00000100:00100000:3.0:1787826277.836932:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11002 00000100:00100000:3.0:1787826277.836950:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c80850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:15672:x1874663332528256:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826277.836970:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826277.837006:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49799 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f63 expref: 12 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.837031:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826277.837046:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f497fb lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f86 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.837059:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:3.0:1787826277.837083:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f497b5 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f6a expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.837094:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:3.0:1787826277.837134:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826277.837235:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c80850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:15672:x1874663332528256:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 291us (1988us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.837243:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11002 00000100:00100000:3.0:1787826277.837327:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11003 00000100:00100000:3.0:1787826277.837336:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:34879:x1874663332528384:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826277.837344:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826277.837356:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4961f lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1db4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f16 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826277.837366:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db4:0x0].0x0 (ffff9f7952ca9500) 00010000:00010000:3.0:1787826277.837385:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826277.838341:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332528512 00000100:00100000:1.0:1787826277.838354:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332528512 00000100:00100000:1.0:1787826277.838463:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11004 00000100:00100000:1.0:1787826277.838475:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:49149:x1874663332528512:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826277.838492:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826277.839206:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49825 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f94 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.839232:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f7845cb1000) 00010000:00010000:1.0:1787826277.839251:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f497c3 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f71 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.839262:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:1.0:1787826277.839271:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4957e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e40:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ef3 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826277.839279:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e40:0x0].0x0 (ffff9f7961d81a00) 00010000:00010000:1.0:1787826277.839288:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826277.839353:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:49149:x1874663332528512:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 881us (3773us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826277.839362:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11004 00000100:00100000:3.0:1787826277.841925:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:34879:x1874663332528384:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 4572us (6542us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826277.841947:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11003 00010000:00010000:0.0:1787826278.572605:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826278.634728:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826278.634787:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826278.634820:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826278.634853:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826278.634924:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826278.634953:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826278.634976:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826278.635003:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826278.635041:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332528768 00000100:00100000:3.0:1787826278.635053:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332528768 00000100:00100000:2.0:1787826278.635320:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332529280 00000100:00100000:2.0:1787826278.635325:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332529280 00000100:00100000:1.0:1787826278.635422:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332529024 00000100:00100000:1.0:1787826278.635427:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332529024 00000100:00100000:1.0:1787826278.635477:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332529536 00000100:00100000:1.0:1787826278.635481:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332529536 00000100:00100000:1.0:1787826278.635505:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332529408 00000100:00100000:1.0:1787826278.635508:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332529408 00000100:00100000:1.0:1787826278.635533:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8763 00000100:00100000:1.0:1787826278.635547:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332529024:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826278.635649:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332529024:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 104us (860us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826278.635658:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8763 00000100:00100000:1.0:1787826278.635679:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8764 00000100:00100000:1.0:1787826278.635687:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332529536:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826278.635714:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332529536:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (861us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826278.635720:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8764 00000100:00100000:1.0:1787826278.635732:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8765 00000100:00100000:1.0:1787826278.635739:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332529408:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826278.635763:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332529408:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (760us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826278.635770:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8765 00000100:00100000:2.0:1787826278.638935:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332528896 00000100:00100000:2.0:1787826278.638956:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332528896 00000100:00100000:2.0:1787826278.639005:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332529152 00000100:00100000:2.0:1787826278.639009:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332529152 00000100:00100000:2.0:1787826278.639056:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14231 00000100:00100000:2.0:1787826278.639071:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332528768:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826278.639193:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332528768:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 125us (4481us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826278.639202:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14231 00000100:00100000:2.0:1787826278.639226:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14232 00000100:00100000:2.0:1787826278.639234:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332529280:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826278.639267:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332529280:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 35us (4448us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826278.639274:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14232 00000100:00100000:2.0:1787826278.639290:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14233 00000100:00100000:2.0:1787826278.639296:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332528896:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826278.639326:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332528896:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (4373us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826278.639408:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14233 00000100:00100000:2.0:1787826278.639425:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14234 00000100:00100000:2.0:1787826278.639433:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332529152:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826278.639463:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332529152:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 32us (4487us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826278.639470:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14234 00000100:00100000:3.0:1787826278.640933:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332528640 00000100:00100000:3.0:1787826278.640950:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332528640 00000100:00100000:3.0:1787826278.641026:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2208 00000100:00100000:3.0:1787826278.641039:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847802450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332528640:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826278.641651:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847802450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332528640:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 612us (6723us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826278.641661:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2208 00000100:00080000:3.0:1787826278.641732:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826278.641739:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787826279.143547:0:164881:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:0.0:1787826279.143608:0:164881:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000100:00100000:3.0:1787826279.405237:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346119296:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.405295:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346119424:0@lo:400:kworker.0 00000100:00100000:3.0:1787826279.405341:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.405352:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826279.405418:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346119168:0@lo:400:kworker.0 00000100:00100000:1.0:1787826279.405494:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346119296 00000100:00100000:1.0:1787826279.405505:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346119296 00000100:00100000:1.0:1787826279.405575:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23317 00000100:00100000:1.0:1787826279.405586:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346119296:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.407914:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f540a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346119936:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408007:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.408056:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346120448:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408086:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.408164:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1787826279.408173:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346119552:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408201:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.408220:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346120064:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408243:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.408259:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346120576:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408294:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.408351:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346119424 00000100:00100000:0.0:1787826279.408359:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346119424 00000100:00100000:0.0:1787826279.408410:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346119936 00000100:00100000:0.0:1787826279.408413:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346119936 00000100:00100000:0.0:1787826279.408577:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346119552 00000100:00100000:0.0:1787826279.408582:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346119552 00000100:00100000:0.0:1787826279.408611:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346120064 00000100:00100000:0.0:1787826279.408614:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346120064 00000100:00100000:0.0:1787826279.408709:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14235 00000100:00100000:0.0:1787826279.408721:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346119424:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408783:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346119424:12345-0@lo:400:kworker.0 Request processed in 65us (3444us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.408792:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14235 00000100:00100000:0.0:1787826279.408812:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14236 00000100:00100000:0.0:1787826279.408818:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346119936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408840:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346119936:12345-0@lo:400:kworker.0 Request processed in 23us (834us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.408846:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14236 00000100:00100000:0.0:1787826279.408855:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14237 00000100:00100000:0.0:1787826279.408860:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346119552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408894:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346119552:12345-0@lo:400:kworker.0 Request processed in 34us (692us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.408900:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14237 00000100:00100000:0.0:1787826279.408910:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14238 00000100:00100000:0.0:1787826279.408916:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346120064:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.408939:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346120064:12345-0@lo:400:kworker.0 Request processed in 24us (695us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.408944:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14238 00000100:00100000:0.0:1787826279.408968:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346120448 00000100:00100000:0.0:1787826279.408972:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346120448 00000100:00100000:0.0:1787826279.409002:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346120576 00000100:00100000:0.0:1787826279.409004:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346120576 00000100:00100000:0.0:1787826279.409023:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8766 00000100:00100000:0.0:1787826279.409028:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346120448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.409056:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346120448:12345-0@lo:400:kworker.0 Request processed in 28us (970us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.409062:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8766 00000100:00100000:0.0:1787826279.409072:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8767 00000100:00100000:0.0:1787826279.409077:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346120576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.409102:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346120576:12345-0@lo:400:kworker.0 Request processed in 24us (807us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.409138:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8767 00000100:00100000:0.0:1787826279.409217:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346119552:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.409229:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346120064:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.409237:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976143 gen 2 00000004:00080000:0.0:1787826279.409243:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7850ad4380, transno 4294976143 00000100:00100000:0.0:1787826279.409255:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346120576:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.409278:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346119424:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.409288:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f540a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346119936:0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.409298:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346120448:0@lo:400:kworker.0 00000100:00080000:3.0:1787826279.410288:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826279.410299:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826279.410304:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826279.410307:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826279.410331:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346119808:0@lo:400:kworker.0 00000100:00100000:3.0:1787826279.410400:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826279.410512:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346119808 00000100:00100000:3.0:1787826279.410523:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346119808 00000100:00100000:3.0:1787826279.410598:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8768 00000100:00100000:3.0:1787826279.410608:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346119808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826279.411292:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346119808:12345-0@lo:400:kworker.0 Request processed in 682us (889us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826279.411304:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8768 00000100:00080000:3.0:1787826279.411336:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826279.411380:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973834 gen 2 00000004:00080000:3.0:1787826279.411386:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7977e0f480, transno 4294973834 00000100:00100000:3.0:1787826279.411401:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346119808:0@lo:400:kworker.0 00000100:00100000:3.0:1787826279.411413:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346120320:0@lo:400:kworker.0 00000100:00100000:3.0:1787826279.411447:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.411747:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346120320 00000100:00100000:0.0:1787826279.411756:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346120320 00000100:00100000:0.0:1787826279.411811:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23318 00000100:00100000:0.0:1787826279.411821:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346120320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826279.415306:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826279.415377:0:30904:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346119168 00000100:00100000:2.0:1787826279.415390:0:30904:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346119168 00000100:00100000:1.0:1787826279.416506:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346119296:12345-0@lo:400:kworker.0 Request processed in 10912us (11156us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826279.416522:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23317 00000100:00100000:0.0:1787826279.416768:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346120320:12345-0@lo:400:kworker.0 Request processed in 4947us (5318us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.416778:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23318 00000100:00100000:2.0:1787826279.416877:0:30904:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2209 00000100:00100000:2.0:1787826279.416893:0:30904:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847801050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346119168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826279.416964:0:30904:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847801050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346119168:12345-0@lo:400:kworker.0 Request processed in 78us (1668us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826279.416970:0:30904:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2209 00000100:00100000:2.0:1787826279.417065:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346119168:0@lo:400:kworker.0 00000100:00100000:2.0:1787826279.417081:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346119680:0@lo:400:kworker.0 00000100:00100000:2.0:1787826279.417137:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826279.417178:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346120192:0@lo:400:kworker.0 00000100:00100000:2.0:1787826279.417208:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826279.417235:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346120704:0@lo:400:kworker.0 00000100:00100000:2.0:1787826279.417262:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.417441:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346119680 00000100:00100000:0.0:1787826279.417454:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346119680 00000100:00100000:0.0:1787826279.417520:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346120704 00000100:00100000:0.0:1787826279.417525:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346120704 00000100:00100000:0.0:1787826279.417550:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14239 00000100:00100000:0.0:1787826279.417558:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346119680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.417602:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346119680:12345-0@lo:400:kworker.0 Request processed in 45us (464us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.417609:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14239 00000100:00100000:0.0:1787826279.417625:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14240 00000100:00100000:0.0:1787826279.417631:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346120704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826279.417672:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346120704:12345-0@lo:400:kworker.0 Request processed in 41us (408us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.417679:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14240 00000100:00100000:1.0:1787826279.417890:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346120192 00000100:00100000:1.0:1787826279.417895:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346120192 00000100:00100000:1.0:1787826279.417945:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8769 00000100:00100000:1.0:1787826279.417954:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346120192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826279.417997:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346120192:12345-0@lo:400:kworker.0 Request processed in 45us (788us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826279.418003:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8769 00000100:00080000:2.0:1787826279.418257:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826279.418264:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826279.418301:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346119680:0@lo:400:kworker.0 00000100:00100000:2.0:1787826279.418313:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346120192:0@lo:400:kworker.0 00000100:00100000:2.0:1787826279.418323:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346120704:0@lo:400:kworker.0 00000100:00100000:3.0:1787826279.418622:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346120320:0@lo:400:kworker.0 00000100:00100000:3.0:1787826279.418765:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346119296:0@lo:400:kworker.0 00010000:00010000:1.0:1787826279.603618:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787826279.917554:0:9993:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8092129443695ns, next at 8093129439845ns, rc = 0 00000100:00100000:0.0:1787826279.917656:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826279.917684:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b618380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346120832:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826279.917779:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826279.917846:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346120832 00000100:00100000:0.0:1787826279.917856:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346120832 00000100:00100000:0.0:1787826279.917935:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6199 00000100:00100000:0.0:1787826279.917957:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346120832:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826279.917998:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3914711040 avail 3683188736 00000100:00100000:0.0:1787826279.918050:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346120832:12345-0@lo:13:osp-pre-1-1.0 Request processed in 105us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826279.918058:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6199 00000100:00100000:0.0:1787826279.918149:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b618380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346120832:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826280.428481:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346120960:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826280.428574:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826280.428947:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346120960 00000100:00100000:2.0:1787826280.428958:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346120960 00000100:00100000:2.0:1787826280.429011:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6200 00000100:00100000:2.0:1787826280.429020:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346120960:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826280.429045:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3914711040 avail 3683188736 00000100:00100000:2.0:1787826280.429089:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346120960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (517us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826280.429127:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6200 00000100:00100000:2.0:1787826280.429179:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346120960:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826280.429335:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346121088:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826280.429379:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826280.429426:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346121088 00000100:00100000:2.0:1787826280.429430:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346121088 00000100:00100000:2.0:1787826280.429471:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23319 00000100:00100000:2.0:1787826280.429481:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346121088:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826280.429519:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442824192 avail 1296584704 00000100:00100000:2.0:1787826280.429567:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346121088:12345-0@lo:41:osp-pre-0-1.0 Request processed in 89us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826280.429576:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23319 00000100:00100000:2.0:1787826280.429609:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346121344:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826280.429621:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b618a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346121216:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826280.429648:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826280.429671:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826280.430000:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346121344 00000100:00100000:2.0:1787826280.430007:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346121344 00000100:00100000:2.0:1787826280.430031:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346121216 00000100:00100000:2.0:1787826280.430034:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346121216 00000100:00100000:2.0:1787826280.430049:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6201 00000100:00100000:2.0:1787826280.430055:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346121344:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1787826280.430080:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826280.430149:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346121344:12345-0@lo:13:osp-pre-0-1.0 Request processed in 93us (500us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826280.430156:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6201 00000100:00100000:2.0:1787826280.430170:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6202 00000100:00100000:2.0:1787826280.430177:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346121216:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826280.430188:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826280.430221:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346121216:12345-0@lo:13:osp-pre-0-0.0 Request processed in 44us (550us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826280.430228:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6202 00000100:00100000:2.0:1787826280.430282:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b618a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346121216:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826280.430861:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346121344:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826280.430917:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346121088:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826280.432704:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826280.432735:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346121472:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826280.432824:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826280.432898:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346121472 00000100:00100000:3.0:1787826280.432908:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346121472 00000100:00100000:3.0:1787826280.432969:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23320 00000100:00100000:3.0:1787826280.432979:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346121472:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787826280.433020:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443426304 avail 1297186816 00000100:00100000:3.0:1787826280.433070:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346121472:12345-0@lo:41:osp-pre-1-0.0 Request processed in 93us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826280.433079:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23320 00000100:00100000:3.0:1787826280.433172:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346121472:0@lo:41:osp-pre-1-0.0 00010000:00010000:3.0:1787826280.623018:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826281.646396:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826281.913696:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826281.914351:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332529664 00000100:00100000:2.0:1787826281.914361:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332529664 00000100:00080000:0.0:1787826281.915158:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826281.915688:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68237 00000100:00100000:2.0:1787826281.915704:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223794:x1874663332529664:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826281.915727:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826281.915749:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4983a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fa2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.915814:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.915827:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.915839:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826281.915891:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826281.915895:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826281.915916:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.915942:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4983a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.915959:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.915968:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826281.915971:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826281.915975:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826281.916044:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223794:x1874663332529664:12345-192.168.203.56@tcp:101:cat.0 Request processed in 345us (2358us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826281.916051:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68237 00000100:00100000:2.0:1787826281.923645:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826281.925910:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332529792 00000100:00100000:0.0:1787826281.925922:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332529792 00000100:00100000:0.0:1787826281.925996:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68238 00000100:00100000:0.0:1787826281.926007:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223794:x1874663332529792:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826281.926027:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826281.926048:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49848 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fa9 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926147:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926822:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926838:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926874:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926883:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926890:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926966:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4985d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926975:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4985d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.926983:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4985d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826281.927011:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4985d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927026:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927033:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927040:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927048:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927054:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826281.927058:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927066:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927078:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826281.927082:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826281.927086:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927092:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4984f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927745:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927758:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927764:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927770:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826281.927774:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927781:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927790:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57af00) 00010000:00010000:0.0:1787826281.927793:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826281.927796:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927801:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49856 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927846:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49848 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa9 expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927870:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4985d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa9 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.927878:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826281.927880:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826281.927883:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826281.927999:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223794:x1874663332529792:12345-192.168.203.56@tcp:101:cat.0 Request processed in 1993us (4363us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826281.928008:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68238 00000100:00100000:1.0:1787826281.934889:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826281.934987:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332529920 00000100:00100000:2.0:1787826281.934997:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332529920 00000100:00100000:2.0:1787826281.935076:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68239 00000100:00100000:2.0:1787826281.935089:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223794:x1874663332529920:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826281.935311:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826281.935334:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49864 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fb0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.935569:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4986b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.935585:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4986b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.935599:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4986b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826281.935691:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e42:0x0]: returning size 4096 00000004:00200000:2.0:1787826281.935696:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826281.935734:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49864 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb0 expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.935752:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4986b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb0 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826281.935761:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:2.0:1787826281.935764:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826281.935767:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826281.935831:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223794:x1874663332529920:12345-192.168.203.56@tcp:101:cat.0 Request processed in 744us (952us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826281.935839:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68239 00000100:00080000:3.0:1787826281.936099:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826281.940748:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826281.940891:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332530048 00000100:00100000:1.0:1787826281.940901:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332530048 00000100:00100000:1.0:1787826281.940984:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68240 00000100:00100000:1.0:1787826281.940994:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223794:x1874663332530048:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826281.941014:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826281.941033:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f49872 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fb7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.941090:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.941136:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.941149:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826281.941201:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18f:0x0]: returning size 4096 00000004:00200000:1.0:1787826281.941205:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826281.941250:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f49872 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 14 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.941267:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.941276:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:1.0:1787826281.941279:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826281.941282:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826281.941341:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223794:x1874663332530048:12345-192.168.203.56@tcp:101:cat.0 Request processed in 348us (1245us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826281.941352:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68240 00000100:00100000:2.0:1787826281.952632:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826281.953924:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332530176 00000100:00100000:1.0:1787826281.953935:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332530176 00000100:00100000:1.0:1787826281.953994:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68241 00000100:00100000:1.0:1787826281.954004:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223794:x1874663332530176:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826281.954023:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826281.954046:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49880 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fbe expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.954474:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.954489:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.954501:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826281.954551:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18d:0x0]: returning size 4096 00000004:00200000:1.0:1787826281.954557:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826281.954595:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49880 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.954612:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.954619:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:1.0:1787826281.954622:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826281.954626:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826281.954728:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223794:x1874663332530176:12345-192.168.203.56@tcp:101:cat.0 Request processed in 723us (2103us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826281.954735:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68241 00000100:00100000:1.0:1787826281.962931:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826281.963018:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332530432 00000100:00100000:0.0:1787826281.963028:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332530432 00000100:00100000:0.0:1787826281.963100:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68242 00000100:00100000:0.0:1787826281.963143:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223794:x1874663332530432:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826281.963165:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826281.963186:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4988e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fc5 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.963274:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.963290:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.963300:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.963311:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.963319:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.963326:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:0.0:1787826281.963404:0:17813:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000004:00080000:1.0:1787826281.968319:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 399 committed 00010000:00010000:1.0:1787826281.968370:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f496c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826281.968385:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f496c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.968431:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.968437:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826281.968459:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f496b9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f496c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826281.968476:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985336 is committed 00000020:00080000:1.0:1787826281.968490:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f79438ba300 refcount 2 committed 0 result 0 batchid 3393 00000020:00080000:1.0:1787826281.968494:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7975783c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f79438bad80 00000020:00080000:1.0:1787826281.968498:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1496 00000020:00080000:1.0:1787826281.968500:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433dad00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e72ec00 00000020:00080000:1.0:1787826281.968503:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1177 00000001:00080000:1.0:1787826281.968511:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985333 is committed 00000020:00080000:1.0:1787826281.968516:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78500aaf00 refcount 2 committed 0 result 0 batchid 3392 00000020:00080000:1.0:1787826281.968518:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433da600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78500aa780 00000020:00080000:1.0:1787826281.968521:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1495 00000020:00080000:1.0:1787826281.968523:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79433da400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e72f800 00000020:00080000:1.0:1787826281.968526:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1176 00000004:00080000:1.0:1787826281.968532:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 398 committed 00000020:00080000:1.0:1787826281.968536:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7844e810c0 refcount 2 committed 0 result 0 batchid 3391 00000020:00080000:1.0:1787826281.968539:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78448e0200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e81f00 00000020:00080000:1.0:1787826281.968541:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1494 00000020:00080000:1.0:1787826281.968543:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78448e0d00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78525ed600 00000020:00080000:1.0:1787826281.968545:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1175 00000020:00080000:1.0:1787826281.968554:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7852f25a80 refcount 2 committed 0 result 0 batchid 3390 00000100:00100000:2.0:1787826281.968554:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346121728:0@lo:6:osp-syn-1-1.0 00000020:00080000:1.0:1787826281.968556:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7850703cc0 00000020:00080000:1.0:1787826281.968559:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1492 00000020:00080000:1.0:1787826281.968561:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784aaa4a00 00000020:00080000:1.0:1787826281.968563:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1174 00000001:00080000:1.0:1787826281.968568:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985327 is committed 00000020:00080000:1.0:1787826281.968577:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3389 batchid updated 00000100:00100000:1.0:1787826281.968590:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346121600:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787826281.968693:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968703:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968710:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968716:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826281.968720:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968726:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968738:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:0.0:1787826281.968741:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826281.968746:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968752:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49776 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968764:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968769:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968774:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968783:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826281.968786:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968791:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968798:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:0.0:1787826281.968800:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826281.968803:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968808:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4977d lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968822:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968827:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968833:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968838:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826281.968841:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968847:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968870:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x4ec94fd6 (ffff9f7952ca9200) 00010000:00010000:0.0:1787826281.968873:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826281.968876:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.968882:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4981e lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826281.969267:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973836 is committed 00000001:00080000:0.0:1787826281.969274:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973835 is committed 00000100:00100000:1.0:1787826281.969655:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:1.0:1787826281.969770:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3393 committed batchid 3389 00000004:00080000:1.0:1787826281.969887:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852405f00 with version 6786 00000020:00080000:1.0:1787826281.969900:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3388 cancel update log [0x200000401:0x3:0x0].1490: rc = 0 00000020:00080000:1.0:1787826281.969944:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3388 cancel update log [0x240000400:0x3:0x0].1172: rc = 0 00000004:00080000:1.0:1787826281.969971:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852405d80 with version 6787 00000020:00080000:1.0:1787826281.969974:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3389 cancel update log [0x200000401:0x3:0x0].1491: rc = 0 00000020:00080000:1.0:1787826281.969995:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3389 cancel update log [0x240000400:0x3:0x0].1173: rc = 0 00000020:00080000:1.0:1787826281.970004:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3389 00000004:00080000:1.0:1787826281.970013:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6786 rpc_version 6786 00000020:00100000:1.0:1787826281.970037:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826281.970045:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b61ad80 x1874663346121984/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826281.970064:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346121984:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826281.970102:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826281.970138:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524050c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826281.970158:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346121984 00000100:00100000:1.0:1787826281.970165:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346121984 00000100:00100000:1.0:1787826281.970212:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23321 00000100:00100000:1.0:1787826281.970221:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b619880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346121984:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826281.971704:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b619880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346121984:12345-0@lo:1000:osp_up0-1.0 Request processed in 1477us (1594us total) trans 8589957319 rc 0/0 00000100:00100000:1.0:1787826281.971716:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23321 00000100:00100000:1.0:1787826281.971785:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346121984:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826281.971796:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6787 rpc_version 6787 00000020:00100000:1.0:1787826281.971814:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826281.971819:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b618a80 x1874663346122240/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826281.971837:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b618a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346122240:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826281.971884:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826281.971912:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524050c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826281.971926:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346122240 00000100:00100000:1.0:1787826281.971931:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346122240 00000100:00100000:1.0:1787826281.971961:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23322 00000100:00100000:1.0:1787826281.971969:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346122240:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826281.972055:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346122240:12345-0@lo:1000:osp_up0-1.0 Request processed in 81us (165us total) trans 8589957320 rc 0/0 00000100:00100000:1.0:1787826281.972063:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23322 00000100:00100000:1.0:1787826281.972100:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b618a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346122240:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826281.973629:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346121600 00000100:00100000:0.0:1787826281.973642:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346121600 00000100:00100000:0.0:1787826281.973708:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11005 00000100:00100000:0.0:1787826281.973717:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3655:x1874663346121600:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787826281.973732:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826281.973757:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f496c0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18c:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f496b9 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.973777:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18c:0x0].0x5a734ce9 (ffff9f7952ca9e00) 00010000:00010000:0.0:1787826281.973790:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826281.973849:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346121600:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 133us (4197us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826281.973867:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11005 00000100:00100000:1.0:1787826281.974088:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346121600:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787826281.977563:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826281.977684:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346121728 00000100:00100000:0.0:1787826281.977694:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346121728 00000100:00100000:0.0:1787826281.977767:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30782 00000100:00100000:0.0:1787826281.977778:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346121728:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1787826281.977812:0:17826:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1223 count 1 00010000:00010000:0.0:1787826281.977837:0:17826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826281.977922:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787826281.977956:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.977967:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4c7:0x0].0x0 since=11s v=(0 0) 00010000:00010000:0.0:1787826281.977971:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826281.977976:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.978010:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.978023:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 6 pid: 35097 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826281.978039:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346122368:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826281.978141:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826281.982624:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346122368:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826281.982658:0:17826:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826281.990601:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826281.990787:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332530560 00000100:00100000:0.0:1787826281.990797:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332530560 00010000:00010000:0.0:1787826281.990863:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 7 pid: 35097 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826281.990901:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21855 00000100:00100000:0.0:1787826281.990912:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332530560:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826281.990927:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826281.990934:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 7 pid: 35097 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.990948:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 7 pid: 35097 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.990973:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b423cb53/0xe21309aa81f494dd lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800eb4 expref: 7 pid: 35097 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.990987:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826281.991004:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826281.991010:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.991020:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826281.991586:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826281.991829:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332530560:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 731us (1049us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826281.991839:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21855 00000100:00080000:1.0:1787826281.992242:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787826281.992266:0:17826:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992279:0:17826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992290:0:17826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992298:0:17826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992317:0:17826:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826281.992531:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346121728:12345-0@lo:6:osp-syn-1-1.0 Request processed in 14754us (14974us total) trans 4294976145 rc 0/0 00000100:00100000:1.0:1787826281.992540:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30782 00010000:00010000:1.0:1787826281.992564:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826281.992571:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992579:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826281.992583:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992591:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992605:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992613:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f498a3 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826281.992632:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826281.992634:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826281.992636:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787826281.992671:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784d98c380/2, rc 0, transno 4294976145 00000100:00100000:1.0:1787826281.992677:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346121728:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1787826282.432655:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826282.432833:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332530688 00000100:00100000:2.0:1787826282.432845:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332530688 00000100:00080000:1.0:1787826282.433272:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826282.433291:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68243 00000100:00100000:2.0:1787826282.433306:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223796:x1874663332530688:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826282.433327:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826282.433348:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f498aa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fcc expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.433568:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.433584:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.433595:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826282.433637:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826282.433641:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826282.433660:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.433684:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f498aa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.433703:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.433711:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826282.433714:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826282.433717:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826282.433777:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223796:x1874663332530688:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 474us (1133us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826282.435097:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68243 00000100:00100000:0.0:1787826282.439749:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826282.439837:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332530816 00000100:00100000:2.0:1787826282.439846:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332530816 00000100:00100000:2.0:1787826282.439974:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68244 00000100:00100000:2.0:1787826282.439987:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223796:x1874663332530816:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826282.440009:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826282.440030:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f498b8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fd3 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.440094:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.440955:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.440972:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.440992:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441000:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441007:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441076:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441085:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441091:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826282.441147:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441162:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441170:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441175:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441182:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441187:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826282.441192:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441199:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441210:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826282.441213:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826282.441218:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441225:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f498bf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441240:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441245:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441250:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441255:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826282.441259:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441265:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441273:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfea00) 00010000:00010000:2.0:1787826282.441275:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826282.441278:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441285:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f498c6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441322:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f498b8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441337:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.441345:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:2.0:1787826282.441347:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826282.441350:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826282.441474:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223796:x1874663332530816:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1488us (1734us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826282.441482:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68244 00000100:00100000:1.0:1787826282.450821:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826282.451528:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332530944 00000100:00100000:3.0:1787826282.451540:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332530944 00000100:00100000:3.0:1787826282.451623:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68245 00000100:00100000:3.0:1787826282.451636:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:223796:x1874663332530944:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826282.451657:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826282.451680:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f498d4 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fda expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.451749:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f498db lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.451762:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f498db lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.451774:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f498db lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826282.451879:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e42:0x0]: returning size 4096 00000004:00200000:3.0:1787826282.451884:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826282.451921:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f498d4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fda expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.451939:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f498db lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fda expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.451949:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:3.0:1787826282.451952:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826282.451955:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00080000:1.0:1787826282.452072:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826282.452152:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223796:x1874663332530944:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 518us (1339us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826282.452159:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68245 00000100:00100000:0.0:1787826282.458450:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826282.459749:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332531072 00000100:00100000:1.0:1787826282.459763:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332531072 00000100:00100000:1.0:1787826282.459835:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68246 00000100:00100000:1.0:1787826282.459846:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223796:x1874663332531072:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826282.459876:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826282.459900:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f498e2 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fe1 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.459969:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.459983:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.459994:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.460010:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.460017:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.460023:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.460099:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f498f7 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461000:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f498f7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461011:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f498f7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826282.461059:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e43:0x0]: returning size 6 00000004:00200000:1.0:1787826282.461063:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826282.461081:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f498f7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461093:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f498f7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461201:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461210:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461216:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461223:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461229:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826282.461234:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461241:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461252:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:1.0:1787826282.461256:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826282.461260:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461267:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f498e9 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461281:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461287:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461293:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461298:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826282.461301:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461307:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461314:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x4ec94fd6 (ffff9f7845cb1d00) 00010000:00010000:1.0:1787826282.461316:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826282.461319:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461324:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f498f0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461360:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f498e2 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fe1 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461507:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f498f7 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fe1 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.461515:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:1.0:1787826282.461517:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826282.461519:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826282.461719:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223796:x1874663332531072:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1874us (3278us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.461728:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68246 00000100:00100000:2.0:1787826282.468789:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826282.469825:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332531200 00000100:00100000:0.0:1787826282.469837:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332531200 00000100:00100000:1.0:1787826282.470187:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68247 00000100:00100000:1.0:1787826282.470201:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223796:x1874663332531200:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826282.470222:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826282.470249:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f498fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fe8 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470319:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470334:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470345:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470356:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470569:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470579:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470669:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49913 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470678:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49913 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470686:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49913 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826282.470726:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db5:0x0]: returning size 6 00000004:00200000:1.0:1787826282.470731:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826282.470749:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49913 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470762:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49913 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470772:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470778:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470784:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470796:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470802:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826282.470807:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470814:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470826:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:1.0:1787826282.470829:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826282.470834:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470840:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49905 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470864:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470871:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470876:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470880:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826282.470883:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470892:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470900:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f794b57a600) 00010000:00010000:1.0:1787826282.470902:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826282.470904:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470909:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4990c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470938:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f498fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fe8 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470953:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49913 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fe8 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.470960:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f7845cb1c00) 00010000:00010000:1.0:1787826282.470963:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826282.470965:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826282.471040:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223796:x1874663332531200:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 842us (2260us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.471049:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68247 00000100:00100000:2.0:1787826282.476832:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826282.477461:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826282.477549:0:6542:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332531584 00000100:00100000:3.0:1787826282.477558:0:6542:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332531584 00000100:00100000:1.0:1787826282.477571:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332531328 00000100:00100000:1.0:1787826282.477575:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332531328 00000100:00100000:0.0:1787826282.477770:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11006 00000100:00100000:0.0:1787826282.477782:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332531328:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:0.0:1787826282.477797:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826282.477819:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f498f7 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fe1 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.477836:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:0.0:1787826282.477845:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826282.477918:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332531328:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 137us (1094us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.477925:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11006 00000100:00100000:1.0:1787826282.478008:0:6542:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1224 00000100:00100000:1.0:1787826282.478017:0:6542:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223796:x1874663332531584:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:1.0:1787826282.478061:0:6542:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826282.478076:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49913 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fe8 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.478088:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f7845cb1c00) 00010000:00010000:1.0:1787826282.478094:0:6542:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826282.478214:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346122496:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.478260:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826282.478290:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524056c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826282.478496:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346122496 00000100:00100000:0.0:1787826282.478502:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346122496 00000100:00100000:0.0:1787826282.478542:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23323 00000100:00100000:0.0:1787826282.478550:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346122496:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.478624:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346122496:12345-0@lo:1000:mdt_io00_002.0 Request processed in 75us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.478631:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23323 00000100:00100000:0.0:1787826282.478663:0:6542:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985327 gen 2 00000004:00080000:0.0:1787826282.478669:0:6542:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985326 committed transno 4294985327 00000020:00080000:0.0:1787826282.478675:0:6542:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7846153180 refcount 2 committed 0 result 0 batchid 4294979076 00000020:00080000:0.0:1787826282.478786:0:6542:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796efc3200 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7846153840 00000020:00080000:0.0:1787826282.478790:0:6542:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].932 00000020:00080000:0.0:1787826282.478792:0:6542:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796efc3000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784837e800 00000020:00080000:0.0:1787826282.478794:0:6542:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].929 00000004:00080000:0.0:1787826282.478825:0:6542:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985327 committed transno 4294985327 00000100:00100000:0.0:1787826282.478840:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346122496:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.478909:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346122624:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.478938:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826282.478956:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826282.478970:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346122624 00000100:00100000:0.0:1787826282.478974:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346122624 00000100:00100000:0.0:1787826282.479012:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23324 00000100:00100000:0.0:1787826282.479019:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346122624:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.479085:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346122624:12345-0@lo:1000:mdt_io00_002.0 Request processed in 67us (147us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.479092:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23324 00000100:00100000:0.0:1787826282.479150:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346122624:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826282.479173:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18f:0x0] 00000100:00100000:0.0:1787826282.479201:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346122752:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.479224:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826282.479239:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784def1840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826282.479250:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346122752 00000100:00100000:0.0:1787826282.479253:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346122752 00000100:00100000:0.0:1787826282.479276:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23325 00000100:00100000:0.0:1787826282.479281:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346122752:12345-0@lo:1000:mdt_io00_002.0 00000020:00080000:1.0:1787826282.479396:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979076 committed batchid 4294979075 00000020:00080000:1.0:1787826282.479450:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979075 00000100:00100000:0.0:1787826282.479719:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346122752:12345-0@lo:1000:mdt_io00_002.0 Request processed in 395us (452us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.479727:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23325 00000100:00100000:1.0:1787826282.479901:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346122752:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826282.479988:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e42:0x0] 00000100:00100000:0.0:1787826282.480017:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346122880:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.480044:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826282.480097:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.480262:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346122880 00000100:00100000:1.0:1787826282.480267:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346122880 00000100:00100000:0.0:1787826282.480476:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23326 00000100:00100000:0.0:1787826282.480485:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346122880:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.480556:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346122880:12345-0@lo:1000:mdt_io00_002.0 Request processed in 72us (510us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.480562:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23326 00000100:00100000:2.0:1787826282.480627:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346122880:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787826282.480665:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18f:0x0] 00000100:00100000:2.0:1787826282.480704:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98e680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346123008:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.480737:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826282.481054:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.481165:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346123008 00000100:00100000:1.0:1787826282.481170:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346123008 00000100:00100000:2.0:1787826282.481249:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23327 00000100:00100000:2.0:1787826282.481257:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346123008:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.481327:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346123008:12345-0@lo:1000:mdt_io00_002.0 Request processed in 70us (590us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826282.481334:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23327 00000100:00100000:2.0:1787826282.481358:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98e680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346123008:0@lo:1000:mdt_io00_002.0 00000004:00020000:2.0:1787826282.481388:0:6542:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:2.0:1787826282.481433:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.481442:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.481452:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 3/0,1 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826282.481464:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346123136:0@lo:101:mdt_io00_002.0 00000100:00100000:2.0:1787826282.481488:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826282.481503:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826282.481513:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346123136 00000100:00100000:2.0:1787826282.481516:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346123136 00000100:00100000:2.0:1787826282.481531:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23328 00000100:00100000:2.0:1787826282.481536:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346123136:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787826282.481545:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826282.481558:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49921 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4991a expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.481582:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.481591:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49921 lrc: 4/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4991a expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.481611:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.481623:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 6 pid: 35092 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826282.481634:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346123264:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826282.481810:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153d80 going to sleep for 11 seconds 00000004:00020000:0.0:1787826282.481928:0:17813:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4482 00010000:00010000:0.0:1787826282.481991:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482001:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482011:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826282.482022:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db5:0x0]: returning size 6 00010000:00010000:0.0:1787826282.482100:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482289:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482352:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482359:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482365:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826282.482370:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482379:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482391:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:0.0:1787826282.482395:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826282.482399:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482407:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f49895 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482421:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482427:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482433:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482437:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826282.482440:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482447:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482454:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f794b57a600) 00010000:00010000:0.0:1787826282.482457:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826282.482460:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482465:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4989c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482507:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4988e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 12 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482521:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.482528:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f794b57af00) 00010000:00010000:0.0:1787826282.482530:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826282.482532:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826282.482582:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223794:x1874663332530432:12345-192.168.203.56@tcp:101:cat.0 Request processed in 519442us (519660us total) trans 8589957321 rc 0/0 00000100:00100000:0.0:1787826282.482590:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68242 00000100:00100000:2.0:1787826282.486637:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826282.486688:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826282.486766:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332531840 00000100:00100000:1.0:1787826282.486775:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332531840 00010000:00010000:1.0:1787826282.486818:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 7 pid: 35092 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826282.486870:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21856 00000100:00100000:1.0:1787826282.486882:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2395:x1874663332531840:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826282.486895:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826282.486903:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 7 pid: 35092 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.486920:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 7 pid: 35092 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.486943:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:1.0:1787826282.486950:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49921 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4991a expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.486961:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49921 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4991a expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.486970:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49921 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4991a expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.487001:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49921 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4991a expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826282.487016:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:8459:1874663346123392:0@lo:105: 00000100:00100000:1.0:1787826282.487061:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826282.487098:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.487143:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346123392 00000100:00100000:1.0:1787826282.487147:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346123392 00000100:00100000:1.0:1787826282.487175:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7815 00000100:00100000:1.0:1787826282.487182:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:8459:x1874663346123392:12345-0@lo:105: 00010000:00010000:1.0:1787826282.487193:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.487207:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.487215:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 4/0,1 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.487221:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.487228:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.487235:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.487247:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ee600) 00000100:00100000:1.0:1787826282.487280:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:8459:x1874663346123392:12345-0@lo:105: Request processed in 99us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.487286:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7815 00000100:00100000:1.0:1787826282.487331:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:8459:1874663346123392:0@lo:105: 00010000:00010000:1.0:1787826282.487349:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826282.487382:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2395:x1874663332531840:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 503us (753us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.487390:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21856 00010000:00010000:0.0:1787826282.487670:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49879 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb7 expref: 6 pid: 35092 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.487685:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346123264:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826282.487707:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49921 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4991a expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.487715:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:0.0:1787826282.487717:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826282.487720:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:0.0:1787826282.487764:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346123136:12345-0@lo:101:mdt_io00_002.0 Request processed in 6227us (6276us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.487771:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23328 00000100:00100000:2.0:1787826282.487840:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346123136:0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787826282.487864:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826282.487882:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18f:0x0] 00010000:00010000:2.0:1787826282.487914:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.487921:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.487929:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826282.487942:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346123520:0@lo:101:mdt_io00_002.0 00000100:00100000:2.0:1787826282.487967:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826282.487982:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.488098:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346123520 00000100:00100000:1.0:1787826282.488127:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346123520 00000100:00100000:1.0:1787826282.488160:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23329 00000100:00100000:1.0:1787826282.488169:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346123520:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826282.488182:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826282.488196:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4993d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49936 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.488215:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4993d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49936 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.488226:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4993d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49936 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.488233:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x4ec94fd6 (ffff9f7845cb1c00) 00010000:00010000:1.0:1787826282.488235:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826282.488237:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:3.0:1787826282.488412:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332531712 00000100:00100000:3.0:1787826282.488417:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332531712 00000100:00100000:3.0:1787826282.488446:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30783 00000100:00100000:3.0:1787826282.488451:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2392:x1874663332531712:12345-192.168.203.56@tcp:8:cat.0 00010000:00080000:3.0:1787826282.488478:0:8444:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from 398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp t0 exp 0000000000000000 cur 8094 last 0 00000020:00000080:3.0:1787826282.488506:0:8444:0:(genops.c:1385:class_connect()) connect: client 398e0287-d3c7-44dd-b1d9-6b85f698e615, cookie 0xe21309aa81f49944 00002000:00100000:3.0:1787826282.488517:0:8444:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli 398e0287-d3c7-44dd-b1d9-6b85f698e615/ffff9f784407c000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113950 ocd_grant: 8437760 ocd_index: 0 ocd_group 0 00002000:00100000:3.0:1787826282.488535:0:8444:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:3.0:1787826282.488602:0:8444:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:3.0:1787826282.488632:0:8444:0:(import.c:65:import_set_state_nolock()) ffff9f784407b800 : changing import state from RECOVER to FULL 00000100:00100000:3.0:1787826282.488673:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2392:x1874663332531712:12345-192.168.203.56@tcp:8:cat.0 Request processed in 220us (1994us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826282.488691:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30783 00000100:00100000:1.0:1787826282.488995:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346123520:12345-0@lo:101:mdt_io00_002.0 Request processed in 826us (1026us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.489002:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23329 00000100:00100000:2.0:1787826282.491962:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346123520:0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787826282.491998:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4993d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.492638:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4993d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826282.492667:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18f:0x0] 00010000:00010000:2.0:1787826282.492763:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.492779:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.492788:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.492829:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.492856:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 10 pid: 35080 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826282.492871:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346123648:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826282.493031:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.499358:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826282.499407:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826282.499518:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346123648:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826282.499547:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.499874:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332531968 00000100:00100000:0.0:1787826282.499881:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332531968 00000100:00100000:0.0:1787826282.499944:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30784 00000100:00100000:0.0:1787826282.499954:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332531968:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826282.499970:0:35352:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826282.499987:0:35352:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f49952 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800fef expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.500043:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346123776:192.168.203.56@tcp:106: 00000100:00100000:0.0:1787826282.500145:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849780000 going to sleep for 11 seconds 00000100:00080000:1.0:1787826282.500271:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826282.500335:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332532096 00000100:00100000:3.0:1787826282.500340:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332532096 00010000:00010000:3.0:1787826282.500363:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 11 pid: 35080 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826282.500542:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21857 00000100:00100000:3.0:1787826282.500552:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c81050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332532096:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826282.500563:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826282.500570:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 11 pid: 35080 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.500582:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 11 pid: 35080 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.500603:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49887 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fbe expref: 11 pid: 35080 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.500619:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:3.0:1787826282.500624:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.500632:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.500639:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.500665:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826282.500695:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c81050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332532096:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 145us (1287us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826282.500702:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21857 00010000:00010000:2.0:1787826282.500783:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.500791:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.500808:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.500822:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.500830:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826282.500890:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346123904:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.500933:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826282.500969:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.501028:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346123904 00000100:00100000:1.0:1787826282.501034:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346123904 00000100:00100000:1.0:1787826282.501072:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23330 00000100:00100000:1.0:1787826282.501080:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346123904:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.501183:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346123904:12345-0@lo:1000:mdt_io00_002.0 Request processed in 104us (249us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.501189:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23330 00000100:00100000:2.0:1787826282.501270:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346123904:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.501317:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124032:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.501341:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826282.501383:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.501788:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346124032 00000100:00100000:1.0:1787826282.501794:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346124032 00000100:00100000:1.0:1787826282.501832:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23331 00000100:00100000:1.0:1787826282.501839:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124032:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.501922:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124032:12345-0@lo:1000:mdt_io00_002.0 Request processed in 83us (580us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.501928:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23331 00000100:00100000:2.0:1787826282.502000:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124032:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.502043:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124160:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.502068:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826282.502091:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.502203:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346124160 00000100:00100000:1.0:1787826282.502207:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346124160 00000100:00100000:1.0:1787826282.502237:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23332 00000100:00100000:1.0:1787826282.502243:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124160:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.502350:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124160:12345-0@lo:1000:mdt_io00_002.0 Request processed in 106us (280us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826282.502357:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23332 00000100:00100000:0.0:1787826282.502584:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124160:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.502648:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124288:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.502679:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826282.502701:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.502744:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346124288 00000100:00100000:1.0:1787826282.502749:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346124288 00000100:00100000:1.0:1787826282.502778:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23333 00000100:00100000:1.0:1787826282.502786:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124288:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.502846:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124288:12345-0@lo:1000:mdt_io00_002.0 Request processed in 60us (165us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.502863:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23333 00000100:00100000:0.0:1787826282.502963:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124288:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.503282:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124416:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826282.503311:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826282.503324:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.503513:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346124416 00000100:00100000:1.0:1787826282.503518:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346124416 00000100:00100000:1.0:1787826282.503691:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23334 00000100:00100000:1.0:1787826282.503700:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124416:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.503773:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124416:12345-0@lo:1000:mdt_io00_002.0 Request processed in 75us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.503782:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23334 00000100:00100000:0.0:1787826282.503834:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124416:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826282.503870:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e42:0x0] 00010000:00010000:0.0:1787826282.503944:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.503964:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.503973:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.504000:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.504014:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 9 pid: 17813 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.504025:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346124544:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826282.504079:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826282.506671:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346124544:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826282.506703:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826282.507069:0:35352:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x2c0000401:0x51d:0x0] updating lvb size: 0 -> 6 00002000:00010000:2.0:1787826282.507073:0:35352:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x2c0000401:0x51d:0x0] updating lvb mtime: 0 -> 1787826276 00002000:00010000:2.0:1787826282.507076:0:35352:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x2c0000401:0x51d:0x0] updating lvb atime: 0 -> 1787826276 00002000:00010000:2.0:1787826282.507078:0:35352:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x2c0000401:0x51d:0x0] updating lvb ctime: 0 -> 1787826276 00002000:00010000:2.0:1787826282.507080:0:35352:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x2c0000401:0x51d:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:2.0:1787826282.507098:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346123776:192.168.203.56@tcp:106: 00010000:00010000:2.0:1787826282.507170:0:35352:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f49952 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fef expref: 7 pid: 35352 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.507197:0:35352:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f49952 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fef expref: 7 pid: 35352 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.507209:0:35352:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:2.0:1787826282.507267:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332531968:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7313us (7917us total) trans 0 rc 301/301 00000100:00100000:2.0:1787826282.507275:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30784 00000100:00100000:3.0:1787826282.509011:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826282.509099:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332532224 00000100:00100000:0.0:1787826282.509150:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332532224 00010000:00010000:0.0:1787826282.509189:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 10 pid: 17813 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.509224:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21858 00000100:00100000:0.0:1787826282.509234:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332532224:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826282.509247:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826282.509254:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 10 pid: 17813 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.509266:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 10 pid: 17813 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.509287:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f49928 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fc5 expref: 10 pid: 17813 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.509302:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f794b57af00) 00010000:00010000:0.0:1787826282.509307:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.509317:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.509323:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.509348:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826282.509404:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332532224:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 171us (400us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.509411:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21858 00010000:00010000:3.0:1787826282.510170:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.510180:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.510236:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.510245:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.510254:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826282.510275:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124672:0@lo:101:mdt_io00_002.0 00000100:00100000:3.0:1787826282.510323:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.510353:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826282.510583:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346124672 00000100:00100000:0.0:1787826282.510590:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346124672 00000100:00100000:0.0:1787826282.510638:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23335 00000100:00100000:0.0:1787826282.510648:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346124672:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826282.510663:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826282.510679:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4996e lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49967 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.510702:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4996e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49967 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.510719:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4996e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49967 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.510728:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:0.0:1787826282.510731:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787826282.510734:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826282.510778:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346124672:12345-0@lo:101:mdt_io00_002.0 Request processed in 132us (454us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.510784:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23335 00000100:00100000:3.0:1787826282.510885:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124672:0@lo:101:mdt_io00_002.0 00010000:00010000:3.0:1787826282.510900:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4996e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.510911:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4996e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826282.510931:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e43:0x0] 00000100:00100000:3.0:1787826282.510976:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124800:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.511005:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.511061:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826282.511643:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346124800 00000100:00100000:2.0:1787826282.511650:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346124800 00000100:00100000:2.0:1787826282.511730:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23336 00000100:00100000:2.0:1787826282.511739:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346124800:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.511893:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346124800:12345-0@lo:1000:mdt_io00_002.0 Request processed in 155us (886us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826282.511901:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23336 00000100:00100000:3.0:1787826282.511958:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124800:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826282.511985:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18f:0x0] 00000100:00100000:3.0:1787826282.512018:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124928:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.512045:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.512323:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.512393:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346124928 00000100:00100000:1.0:1787826282.512399:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346124928 00000100:00100000:1.0:1787826282.512437:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23337 00000100:00100000:1.0:1787826282.512444:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346124928:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.512509:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346124928:12345-0@lo:1000:mdt_io00_002.0 Request processed in 66us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.512516:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23337 00000100:00100000:3.0:1787826282.512645:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346124928:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826282.512663:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e42:0x0] 00010000:00010000:3.0:1787826282.512697:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.512709:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.512718:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826282.512762:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125056:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.512787:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.512808:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.512883:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346125056 00000100:00100000:1.0:1787826282.512887:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346125056 00000100:00100000:1.0:1787826282.512914:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23338 00000100:00100000:1.0:1787826282.512920:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125056:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.512966:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125056:12345-0@lo:1000:mdt_io00_002.0 Request processed in 46us (179us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.512972:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23338 00000100:00100000:3.0:1787826282.513033:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125056:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826282.513049:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e42:0x0] 00000100:00100000:3.0:1787826282.513070:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125184:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.513091:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.513144:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.513242:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346125184 00000100:00100000:1.0:1787826282.513247:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346125184 00000100:00100000:1.0:1787826282.513276:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23339 00000100:00100000:1.0:1787826282.513284:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125184:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.513332:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125184:12345-0@lo:1000:mdt_io00_002.0 Request processed in 49us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.513338:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23339 00000100:00100000:3.0:1787826282.513412:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125184:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.513442:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125312:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.513465:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.513483:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.513584:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346125312 00000100:00100000:1.0:1787826282.513588:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346125312 00000100:00100000:1.0:1787826282.513612:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23340 00000100:00100000:1.0:1787826282.513618:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125312:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.513664:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125312:12345-0@lo:1000:mdt_io00_002.0 Request processed in 46us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.513671:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23340 00000100:00100000:3.0:1787826282.513734:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125312:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826282.513753:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18f:0x0] 00000100:00100000:3.0:1787826282.513773:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125440:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.513794:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.513812:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.513864:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346125440 00000100:00100000:1.0:1787826282.513868:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346125440 00000100:00100000:1.0:1787826282.513894:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23341 00000100:00100000:1.0:1787826282.513900:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125440:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.513945:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125440:12345-0@lo:1000:mdt_io00_002.0 Request processed in 45us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.513951:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23341 00000100:00100000:3.0:1787826282.514703:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125440:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826282.514733:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e42:0x0] 00000100:00100000:3.0:1787826282.514772:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125568:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.514805:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.514833:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.514904:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346125568 00000100:00100000:1.0:1787826282.514909:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346125568 00000100:00100000:1.0:1787826282.514947:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23342 00000100:00100000:1.0:1787826282.514956:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125568:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826282.515017:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125568:12345-0@lo:1000:mdt_io00_002.0 Request processed in 62us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.515024:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23342 00000100:00100000:3.0:1787826282.515210:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125568:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.515245:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125696:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.515270:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.515282:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826282.515398:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346125696 00000100:00100000:2.0:1787826282.515404:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346125696 00000100:00100000:2.0:1787826282.515429:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23343 00000100:00100000:2.0:1787826282.515437:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125696:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.515492:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125696:12345-0@lo:1000:mdt_io00_002.0 Request processed in 56us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826282.515498:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23343 00000100:00100000:3.0:1787826282.515677:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125696:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.515749:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125824:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826282.515774:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.515796:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826282.515861:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346125824 00000100:00100000:2.0:1787826282.515866:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346125824 00000100:00100000:2.0:1787826282.515896:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23344 00000100:00100000:2.0:1787826282.515903:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125824:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826282.515972:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346125824:12345-0@lo:1000:mdt_io00_002.0 Request processed in 68us (195us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826282.515977:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23344 00000100:00100000:1.0:1787826282.516073:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346125824:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826282.516475:0:6542:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1db5:0x0] is inserted into orphan list, open count = 1 00000040:00080000:1.0:1787826282.516542:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].930, 1240 off875152 00000040:00080000:1.0:1787826282.516597:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].933, 1320 off973896 00000004:00080000:1.0:1787826282.516603:0:6542:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7852405240 with version 1814 00000004:00080000:3.0:1787826282.517081:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1814 rpc_version 1814 00000020:00100000:3.0:1787826282.517132:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826282.517142:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b619c00 x1874663346126080/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826282.517160:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346126080:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826282.517205:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.517228:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826282.517568:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826282.517645:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332532352 00000100:00100000:3.0:1787826282.517652:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332532352 00000100:00100000:3.0:1787826282.517704:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68248 00000100:00100000:3.0:1787826282.517714:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223794:x1874663332532352:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826282.517732:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826282.517754:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4997c lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1db5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ff6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.517821:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4997c lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1db5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ff6 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.517835:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4997c lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1db5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ff6 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.517864:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f794b57af00) 00010000:00010000:3.0:1787826282.517868:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787826282.517872:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826282.517919:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223794:x1874663332532352:12345-192.168.203.56@tcp:101:cat.0 Request processed in 206us (352us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826282.517927:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68248 00000100:00100000:2.0:1787826282.519243:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346126080 00000100:00100000:2.0:1787826282.519256:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346126080 00000100:00100000:2.0:1787826282.519312:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23345 00000100:00100000:2.0:1787826282.519321:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346126080:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826282.519777:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346126080:12345-0@lo:1000:osp_up1-0.0 Request processed in 456us (2569us total) trans 4294985337 rc 0/0 00000100:00100000:2.0:1787826282.519788:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23345 00000100:00100000:1.0:1787826282.520247:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346126080:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787826282.520336:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520352:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520361:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520438:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826282.520446:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520453:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520468:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f794b57af00) 00010000:00010000:3.0:1787826282.520471:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826282.520476:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520482:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f49960 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826282.520499:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e43:0x0] 00010000:00010000:3.0:1787826282.520506:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4996e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520513:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4996e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520526:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520535:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520541:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520546:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826282.520549:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520554:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520561:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:3.0:1787826282.520563:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826282.520566:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520571:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4994b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826282.520587:0:6542:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000b575b7f reply state 000000007f58303b transno 8589957322 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520599:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520605:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f794b57a600) 00010000:00010000:3.0:1787826282.520607:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826282.520610:0:6542:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:3.0:1787826282.520613:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18f:0x0] 00010000:00010000:3.0:1787826282.520618:0:6542:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520624:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520631:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520694:0:6542:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826282.520699:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x18f:0x0] 00010000:00010000:3.0:1787826282.520703:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4993d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520710:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4993d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520718:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520724:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520729:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520734:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826282.520737:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520747:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520755:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f79460a2900) 00010000:00010000:3.0:1787826282.520758:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826282.520760:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.520766:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f49975 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826282.520862:0:6542:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:223796:x1874663332531584:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 42844us (43400us total) trans 8589957322 rc 0/0 00000100:00100000:3.0:1787826282.520871:0:6542:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1224 00010000:00010000:1.0:1787826282.521933:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826282.521943:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.521952:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826282.521958:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826282.521965:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.522001:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 2/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.522007:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826282.522967:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826282.523056:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332532480 00000100:00100000:2.0:1787826282.523064:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332532480 00000100:00100000:2.0:1787826282.523146:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30785 00000100:00100000:2.0:1787826282.523156:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223794:x1874663332532480:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826282.523172:0:20167:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826282.523190:0:20167:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760800ffd expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.523215:0:20167:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.523224:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x51d:0x0].0x0 since=5s v=(0 0) 00010000:00010000:2.0:1787826282.523228:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826282.523233:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 7 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.523265:0:20167:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.523278:0:20167:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 5 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826282.523292:0:20167:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20167:1874663346126336:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826282.523385:0:20167:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826282.524772:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346126208:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826282.524832:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826282.524933:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 1/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.524944:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee600/0xe21309aa81f4991a lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49921 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.524968:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826282.524970:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826282.524972:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826282.525817:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346126208 00000100:00100000:2.0:1787826282.525826:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346126208 00000100:00100000:2.0:1787826282.525898:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11007 00000100:00100000:2.0:1787826282.525908:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346126208:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826282.525923:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826282.525957:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49921 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4991a expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.525973:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7952ca9a00) 00010000:00010000:2.0:1787826282.525983:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826282.527504:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826282.527997:0:20167:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20167:1874663346126336:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826282.528036:0:20167:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 7 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.528057:0:20167:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:3.0:1787826282.528152:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223794:x1874663332532480:12345-192.168.203.56@tcp:101:cat.0 Request processed in 4997us (5192us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826282.528162:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30785 00000100:00100000:2.0:1787826282.529163:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346126208:12345-0@lo:103:ldlm_bl.0 Request processed in 3250us (4325us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826282.529174:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11007 00000100:00100000:2.0:1787826282.529212:0:24160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332532608 00000100:00100000:2.0:1787826282.529221:0:24160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332532608 00002000:00100000:2.0:1787826282.529275:0:24160:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x51d:0x0] (0->5) req@ffff9f7849dd1880 x1874663332532608/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:638/0 lens 496/0 e 0 to 0 dl 1787826293 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826282.529290:0:24160:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd1880 with x1874663332532608 ext(0->5) 00010000:00010000:2.0:1787826282.529298:0:24160:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 6 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.529314:0:24160:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 6 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826282.529322:0:24160:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9f7849dd1880 00000100:00100000:2.0:1787826282.529348:0:24160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2488 00000100:00100000:2.0:1787826282.529358:0:24160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332532608:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826282.530480:0:24160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332532608:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1118us (2979us total) trans 4294976146 rc 0/0 00000100:00100000:2.0:1787826282.530490:0:24160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2488 00002000:00100000:2.0:1787826282.530498:0:24160:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_011: refresh rw locks for [0x2c0000401:0x51d:0x0] (0->5) req@ffff9f7849dd1880 x1874663332532608/t4294976146(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:638/0 lens 496/448 e 0 to 0 dl 1787826293 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826282.530506:0:24160:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd1880 with x1874663332532608 ext(0->5) 00010000:00010000:2.0:1787826282.530513:0:24160:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 8 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.530527:0:24160:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 8 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826282.530533:0:24160:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9f7849dd1880 00000100:00100000:0.0:1787826282.530620:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346126208:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826282.532890:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826282.532977:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332532736 00000100:00100000:0.0:1787826282.532986:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332532736 00010000:00010000:0.0:1787826282.533026:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 8 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.533067:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21859 00000100:00100000:0.0:1787826282.533077:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332532736:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826282.533092:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1787826282.533411:0:6512:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000401:0x51d:0x0] updating lvb blocks from disk: 1 -> 8 00010000:00010000:0.0:1787826282.533419:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 8 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.533432:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 8 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.533455:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000071996532/0xe21309aa81f49809 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f8d expref: 8 pid: 24218 timeout: 8194 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.533472:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826282.533476:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826282.533486:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.533496:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.533539:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.533554:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6512:1874663346126464:192.168.203.56@tcp:105: 00000100:00100000:0.0:1787826282.533638:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826282.541822:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6512:1874663346126464:192.168.203.56@tcp:105: 00010000:00010000:0.0:1787826282.541862:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826282.541921:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332532736:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 8845us (9039us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.541933:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21859 00000100:00100000:1.0:1787826282.543316:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826282.543389:0:20174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332532864 00000100:00100000:2.0:1787826282.543400:0:20174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332532864 00002000:00100000:2.0:1787826282.543454:0:20174:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x51d:0x0] (0->4095) req@ffff9f7849dd1f80 x1874663332532864/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:638/0 lens 488/0 e 0 to 0 dl 1787826293 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826282.543464:0:20174:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd1f80 with x1874663332532864 ext(0->4095) 00002000:00010000:2.0:1787826282.543469:0:20174:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7849dd1f80 00000100:00100000:2.0:1787826282.543533:0:20174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27632 00000100:00100000:2.0:1787826282.543545:0:20174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332532864:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:2.0:1787826282.543710:0:20174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332532864:12345-192.168.203.56@tcp:3:cat.0 Request processed in 167us (400us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787826282.543718:0:20174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27632 00002000:00100000:2.0:1787826282.543725:0:20174:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x51d:0x0] (0->4095) req@ffff9f7849dd1f80 x1874663332532864/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:638/0 lens 488/4536 e 0 to 0 dl 1787826293 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826282.543733:0:20174:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd1f80 with x1874663332532864 ext(0->4095) 00002000:00010000:2.0:1787826282.543737:0:20174:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7849dd1f80 00000100:00100000:1.0:1787826282.552678:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826282.554457:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332533120 00000100:00100000:3.0:1787826282.554471:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332533120 00000100:00100000:3.0:1787826282.554596:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9305 00000100:00100000:3.0:1787826282.554608:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223794:x1874663332533120:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:3.0:1787826282.554826:0:6531:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1db5:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1787826282.554915:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].210, 64 off21632 00000004:00080000:3.0:1787826282.554927:0:6531:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8094766820880ns, next at 8095766818005ns, rc = 0 00010000:00010000:3.0:1787826282.554983:0:6531:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.555000:0:6531:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.555012:0:6531:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.555021:0:6531:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.555028:0:6531:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.555034:0:6531:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.555064:0:6531:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826282.555212:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223794:x1874663332533120:12345-192.168.203.56@tcp:35:cat.0 Request processed in 606us (2545us total) trans 8589957323 rc 0/0 00000100:00100000:3.0:1787826282.555221:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9305 00010000:00010000:0.0:1787826282.555864:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826282.555873:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.555886:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826282.555891:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.555899:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.555918:0:30445:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f794b57af00) 00010000:00010000:0.0:1787826282.555921:0:30445:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826282.555926:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.555932:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4998a lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.555946:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826282.555948:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826282.555949:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826282.671898:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826282.869921:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4993d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826282.869941:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4993d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.869992:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4993d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.870000:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826282.870046:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef000/0xe21309aa81f49936 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4993d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.870071:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4996e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826282.870079:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4996e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.870096:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4996e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.870102:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826282.870140:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69800/0xe21309aa81f49967 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4996e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826282.870154:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957323 is committed 00000004:00080000:3.0:1787826282.870160:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 215 committed 00000001:00080000:3.0:1787826282.870190:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957322 is committed 00000100:00100000:0.0:1787826282.870387:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346126592:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826282.870456:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826282.870499:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826282.870505:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346126720:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826282.870528:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826282.870633:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346126592 00000100:00100000:0.0:1787826282.870640:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346126592 00000100:00100000:0.0:1787826282.870682:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346126720 00000100:00100000:0.0:1787826282.870685:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346126720 00000100:00100000:0.0:1787826282.870717:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11008 00000100:00100000:0.0:1787826282.870726:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346126592:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826282.870740:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826282.870761:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4993d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18f:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49936 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.870777:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x4ec94fd6 (ffff9f7845cb1c00) 00010000:00010000:0.0:1787826282.870787:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826282.870822:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346126592:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 96us (366us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.870828:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11008 00000100:00100000:0.0:1787826282.870859:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11009 00000100:00100000:0.0:1787826282.870867:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346126720:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826282.870875:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826282.870887:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4996e lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49967 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.870898:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:0.0:1787826282.870905:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826282.870927:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346126720:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 61us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826282.870933:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11009 00000100:00100000:0.0:1787826282.870948:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826282.870956:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845aff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346126848:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787826282.870985:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826282.871013:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346126848 00000100:00100000:0.0:1787826282.871017:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346126848 00000100:00100000:0.0:1787826282.871054:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30786 00000100:00100000:0.0:1787826282.871061:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346126848:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1787826282.871085:0:8443:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1309 count 1 00010000:00010000:0.0:1787826282.871125:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.871144:0:8443:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.871153:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=1s v=(1 0) 00010000:00010000:0.0:1787826282.871156:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826282.871160:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.871185:0:8443:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.871211:0:8443:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 6 pid: 20167 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.871223:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8443:1874663346126976:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826282.871289:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a600 going to sleep for 11 seconds 00000100:00080000:0.0:1787826282.871303:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826282.871337:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346126720:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826282.871675:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.871684:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.871690:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.871696:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00000100:00100000:0.0:1787826282.871841:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346126592:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826282.872008:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.872021:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.872027:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.872032:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826282.872036:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.872044:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.872058:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f794b57a600) 00010000:00010000:0.0:1787826282.872061:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826282.872065:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.872071:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f497f4 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787826282.872188:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979076 batchid updated 00000001:00080000:3.0:1787826282.872214:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957320 is committed 00000020:00080000:0.0:1787826282.873001:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979076 00000004:00080000:0.0:1787826282.873063:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784548a300 with version 1815 00000020:00080000:0.0:1787826282.873084:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979076 cancel update log [0x240000404:0x2:0x0].932: rc = 0 00010000:00010000:2.0:1787826282.873208:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.873221:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.873234:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f794b57a600) 00010000:00010000:2.0:1787826282.873237:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826282.873481:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826282.873489:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49959 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826282.873641:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1815 rpc_version 1815 00000020:00100000:3.0:1787826282.873675:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826282.873685:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e760e00 x1874663346127232/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826282.873716:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346127232:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826282.873779:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826282.873815:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703540 going to sleep for 11 seconds 00000020:00080000:0.0:1787826282.874174:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979076 cancel update log [0x200000bd0:0x2:0x0].929: rc = 0 00000020:00080000:0.0:1787826282.874186:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979076 00000100:00100000:2.0:1787826282.874259:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346127232 00000100:00100000:2.0:1787826282.874266:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346127232 00000100:00100000:2.0:1787826282.874324:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23346 00000100:00100000:2.0:1787826282.874341:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346127232:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826282.874690:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346127232:12345-0@lo:1000:osp_up1-0.0 Request processed in 356us (910us total) trans 4294985338 rc 0/0 00000100:00100000:2.0:1787826282.874709:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23346 00000100:00100000:0.0:1787826282.875627:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346127232:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826282.875991:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8443:1874663346126976:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826282.876016:0:8443:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826282.884747:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826282.885166:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332533248 00000100:00100000:0.0:1787826282.885177:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332533248 00010000:00010000:0.0:1787826282.885234:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 7 pid: 20167 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826282.885284:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21860 00000100:00100000:0.0:1787826282.885294:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332533248:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826282.885310:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826282.885328:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 7 pid: 20167 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.885343:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 7 pid: 20167 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.885382:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f49983 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760800ffd expref: 7 pid: 20167 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.885399:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826282.885403:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826282.885409:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826282.885419:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.885513:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826282.885579:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332533248:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 285us (840us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826282.885588:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21860 00010000:00010000:3.0:1787826282.887653:0:8443:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.887670:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.887683:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.887692:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826282.887724:0:8443:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826282.887755:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826282.887763:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.887773:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826282.887778:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.887788:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.887809:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.887817:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4ec859c/0xe21309aa81f49991 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826282.887843:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826282.887857:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826282.887859:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826282.888046:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346126848:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16982us (17057us total) trans 4294976147 rc 0/0 00000100:00100000:3.0:1787826282.888055:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30786 00000004:00080000:0.0:1787826282.888179:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7845aff100/2, rc 0, transno 4294976147 00000100:00100000:0.0:1787826282.888187:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845aff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346126848:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1787826283.229689:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826283.229736:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826283.229795:0:8450:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332533376 00000100:00100000:2.0:1787826283.229806:0:8450:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332533376 00002000:00100000:2.0:1787826283.229887:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4ca:0x0] (0->5) req@ffff9f7849dd1c00 x1874663332533376/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:639/0 lens 496/0 e 0 to 0 dl 1787826294 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826283.229900:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd1c00 with x1874663332533376 ext(0->5) 00002000:00010000:2.0:1787826283.229906:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7849dd1c00 00000100:00100000:2.0:1787826283.229937:0:8450:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27633 00000100:00100000:2.0:1787826283.229949:0:8450:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332533376:12345-192.168.203.56@tcp:4:bash.0 00000100:00080000:1.0:1787826283.230609:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826283.230759:0:8450:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332533376:12345-192.168.203.56@tcp:4:bash.0 Request processed in 810us (1078us total) trans 4294973837 rc 0/0 00000100:00100000:2.0:1787826283.230769:0:8450:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27633 00002000:00100000:2.0:1787826283.230777:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4ca:0x0] (0->5) req@ffff9f7849dd1c00 x1874663332533376/t4294973837(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:639/0 lens 496/448 e 0 to 0 dl 1787826294 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826283.230785:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd1c00 with x1874663332533376 ext(0->5) 00002000:00010000:2.0:1787826283.230789:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7849dd1c00 00000100:00100000:0.0:1787826283.231147:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332533504 00000100:00100000:0.0:1787826283.231155:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332533504 00000100:00100000:0.0:1787826283.231198:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30787 00000100:00100000:0.0:1787826283.231209:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332533504:12345-192.168.203.56@tcp:16:kworker.0 00000001:00080000:1.0:1787826283.233514:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973837 is committed 00000100:00100000:0.0:1787826283.238240:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332533504:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 7026us (8496us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.238254:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30787 00000100:00100000:1.0:1787826283.605489:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.605603:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332533632 00000100:00100000:3.0:1787826283.605613:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332533632 00000100:00100000:3.0:1787826283.605682:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68249 00000100:00100000:3.0:1787826283.605823:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:223832:x1874663332533632:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826283.605854:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826283.605879:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49998 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801004 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.605946:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4999f lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.605959:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4999f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.605971:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4999f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826283.606130:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e42:0x0]: returning size 4096 00000004:00200000:3.0:1787826283.606135:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787826283.606155:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4999f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.606182:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49998 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801004 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.606205:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4999f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801004 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.606213:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:3.0:1787826283.606216:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826283.606219:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826283.606276:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223832:x1874663332533632:12345-192.168.203.56@tcp:101:rm.0 Request processed in 586us (807us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.606283:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68249 00000100:00080000:1.0:1787826283.606444:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826283.608379:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.608479:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332533760 00000100:00100000:3.0:1787826283.608490:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332533760 00000100:00100000:3.0:1787826283.608555:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68250 00000100:00100000:3.0:1787826283.608773:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223832:x1874663332533760:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826283.608798:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826283.608828:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f499a6 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080100b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.608897:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.608912:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.608924:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826283.608964:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x18f:0x0]: returning size 4096 00000004:00200000:3.0:1787826283.608968:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826283.609000:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f499a6 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.609013:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.609021:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:3.0:1787826283.609025:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826283.609028:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826283.609088:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223832:x1874663332533760:12345-192.168.203.56@tcp:101:rm.0 Request processed in 523us (715us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.609095:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68250 00000100:00100000:1.0:1787826283.614195:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826283.614239:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332533888 00000100:00100000:1.0:1787826283.614247:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332533888 00000100:00100000:1.0:1787826283.614307:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68251 00000100:00100000:1.0:1787826283.614319:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223832:x1874663332533888:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826283.614341:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826283.614366:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f499b4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801012 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.614431:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.614444:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.614456:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826283.614515:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18d:0x0]: returning size 4096 00000004:00200000:1.0:1787826283.614520:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826283.614558:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f499b4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.614576:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.614583:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:1.0:1787826283.614586:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826283.614589:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826283.614645:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223832:x1874663332533888:12345-192.168.203.56@tcp:101:rm.0 Request processed in 329us (460us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826283.614652:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68251 00000100:00080000:1.0:1787826283.614674:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826283.622653:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826283.622791:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332534144 00000100:00100000:1.0:1787826283.622805:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332534144 00000100:00100000:1.0:1787826283.622876:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68252 00000100:00100000:1.0:1787826283.622893:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223832:x1874663332534144:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826283.622914:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826283.622936:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f499c2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801019 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826283.623097:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e42:0x0]: returning size 4096 00010000:00010000:1.0:1787826283.623223:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f499c2 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801019 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.623238:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7845cb1d00) 00010000:00010000:1.0:1787826283.623240:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826283.623244:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f499c2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801019 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.623258:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826283.623550:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223832:x1874663332534144:12345-192.168.203.56@tcp:101:rm.0 Request processed in 658us (901us total) trans 4294985339 rc 301/301 00000100:00100000:1.0:1787826283.623559:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68252 00000100:00100000:1.0:1787826283.631930:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826283.632185:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332534272 00000100:00100000:2.0:1787826283.632196:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332534272 00000100:00100000:2.0:1787826283.635923:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9306 00000100:00100000:2.0:1787826283.635940:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd1450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223832:x1874663332534272:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826283.639870:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd1450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223832:x1874663332534272:12345-192.168.203.56@tcp:37:rm.0 Request processed in 3924us (7937us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826283.639885:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9306 00000100:00100000:0.0:1787826283.646773:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826283.649213:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332534400 00000100:00100000:1.0:1787826283.649227:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332534400 00000100:00100000:1.0:1787826283.649416:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9307 00000100:00100000:1.0:1787826283.649428:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223832:x1874663332534400:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826283.659032:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223832:x1874663332534400:12345-192.168.203.56@tcp:37:rm.0 Request processed in 9596us (12257us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826283.659045:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9307 00000100:00100000:0.0:1787826283.667776:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826283.667871:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332534656 00000100:00100000:1.0:1787826283.667882:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332534656 00000100:00100000:1.0:1787826283.667947:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9308 00000100:00100000:1.0:1787826283.667960:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:223832:x1874663332534656:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:0.0:1787826283.668833:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1178, 344 off1071704 00000040:00080000:0.0:1787826283.668898:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1497, 424 off1511616 00000004:00080000:0.0:1787826283.668905:0:17601:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78524059c0 with version 6788 00000004:00080000:1.0:1787826283.668973:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6788 rpc_version 6788 00000020:00100000:1.0:1787826283.669003:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826283.669012:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7849dd1500 x1874663346127488/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826283.669036:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346127488:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826283.669095:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826283.669212:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346127488 00000100:00100000:3.0:1787826283.669220:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346127488 00000100:00100000:3.0:1787826283.669259:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23347 00000100:00100000:3.0:1787826283.669269:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346127488:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826283.672535:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346127488:12345-0@lo:1000:osp_up0-1.0 Request processed in 3258us (3430us total) trans 8589957324 rc 0/0 00000100:00100000:3.0:1787826283.672549:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23347 00000100:00100000:1.0:1787826283.674039:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b4df3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826283.674078:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957320 gen 2 00000004:00080000:1.0:1787826283.674083:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957319 committed transno 8589957320 00000004:00080000:1.0:1787826283.674122:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957320 committed transno 8589957320 00000100:00100000:1.0:1787826283.674138:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f7849dd1500 x1874663346127488/t8589957324(8589957324) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826299 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826283.674264:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346127488:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826283.674550:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223832:x1874663332534656:12345-192.168.203.56@tcp:35:rm.0 Request processed in 6590us (6778us total) trans 4294985340 rc 0/0 00000100:00100000:3.0:1787826283.674562:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9308 00000100:00100000:2.0:1787826283.679973:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826283.680034:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332534912 00000100:00100000:1.0:1787826283.680044:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332534912 00000100:00100000:1.0:1787826283.680102:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68253 00000100:00100000:1.0:1787826283.680132:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223832:x1874663332534912:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826283.680165:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826283.680188:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f499d0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801020 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.680335:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f499d7 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.680349:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f499d7 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.680360:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f499d7 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826283.680372:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e42:0x0]: returning size 4096 00010000:00010000:1.0:1787826283.680464:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f499d0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801020 expref: 13 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.680486:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f499d7 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801020 expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.680495:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:1.0:1787826283.680498:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1787826283.680502:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826283.680560:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223832:x1874663332534912:12345-192.168.203.56@tcp:101:rm.0 Request processed in 430us (595us total) trans 4294985341 rc 0/0 00000100:00100000:1.0:1787826283.680568:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68253 00000100:00100000:3.0:1787826283.689988:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826283.690085:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332535040 00000100:00100000:0.0:1787826283.690095:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332535040 00000100:00100000:0.0:1787826283.690253:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68254 00000100:00100000:0.0:1787826283.690265:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223832:x1874663332535040:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:0.0:1787826283.690319:0:35092:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345618944 00000100:00100000:0.0:1787826283.690398:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223832:x1874663332535040:12345-192.168.203.56@tcp:41:rm.0 Request processed in 135us (418us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.690407:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68254 00010000:00010000:0.0:1787826283.692260:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826283.699773:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826283.699879:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332535168 00000100:00100000:2.0:1787826283.699892:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332535168 00000100:00100000:2.0:1787826283.699956:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68255 00000100:00100000:2.0:1787826283.699968:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:223832:x1874663332535168:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826283.699989:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826283.700031:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f499e5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801027 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700100:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700135:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700146:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700161:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700167:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700172:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700227:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f499fa lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700234:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f499fa lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700241:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f499fa lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826283.700266:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f499fa lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700281:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700288:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700299:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700318:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700324:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826283.700328:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700335:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700348:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:2.0:1787826283.700351:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826283.700356:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700362:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f499ec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700375:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700380:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700385:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700390:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826283.700394:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700400:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700406:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f784abfe200) 00010000:00010000:2.0:1787826283.700409:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826283.700412:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700418:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f499f3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700452:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f499e5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801027 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700465:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f499fa lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801027 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.700472:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f784abfe500) 00010000:00010000:2.0:1787826283.700474:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826283.700476:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:2.0:1787826283.700536:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223832:x1874663332535168:12345-192.168.203.56@tcp:101:rm.0 Request processed in 570us (772us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826283.700544:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68255 00000100:00100000:1.0:1787826283.711372:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.711437:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332535296 00000100:00100000:3.0:1787826283.711446:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332535296 00000100:00100000:3.0:1787826283.711507:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68256 00000100:00100000:3.0:1787826283.711518:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:223832:x1874663332535296:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826283.711542:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826283.711565:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f49a01 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080102e expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.711625:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49a08 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.711638:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49a08 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.711650:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49a08 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826283.711698:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e43:0x0]: returning size 6 00000004:00200000:3.0:1787826283.711702:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:3.0:1787826283.711739:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f49a01 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080102e expref: 14 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.711755:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49a08 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080102e expref: 13 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.711764:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f7961d81c00) 00010000:00010000:3.0:1787826283.711767:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826283.711771:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826283.711825:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223832:x1874663332535296:12345-192.168.203.56@tcp:101:rm.0 Request processed in 309us (464us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.711833:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68256 00000100:00100000:3.0:1787826283.725036:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.725075:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.725102:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332535680 00000100:00100000:0.0:1787826283.725252:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332535424 00000100:00100000:0.0:1787826283.725263:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332535424 00000100:00100000:0.0:1787826283.725326:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11010 00000100:00100000:0.0:1787826283.725337:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c81850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332535424:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826283.725354:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826283.725378:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f499fa lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801027 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.725394:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f784abfe500) 00010000:00010000:0.0:1787826283.725405:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826283.725467:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c81850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332535424:12345-192.168.203.56@tcp:103:rm.0 Request processed in 132us (440us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.725476:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11010 00000100:00100000:3.0:1787826283.725657:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332535680 00000100:00100000:3.0:1787826283.725699:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68257 00000100:00100000:3.0:1787826283.725708:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223832:x1874663332535680:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826283.725757:0:35098:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826283.725773:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f49a08 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e43:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080102e expref: 13 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.725791:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f7961d81c00) 00010000:00010000:3.0:1787826283.725799:0:35098:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826283.725865:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.725874:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.725884:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.725906:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e762a00 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346127616:0@lo:101:mdt00_013.0 00000100:00100000:3.0:1787826283.725947:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826283.725979:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3b600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.726019:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346127616 00000100:00100000:3.0:1787826283.726023:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346127616 00000100:00100000:3.0:1787826283.726055:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23348 00000100:00100000:3.0:1787826283.726062:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346127616:12345-0@lo:101:mdt00_013.0 00010000:00010000:3.0:1787826283.726076:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826283.726089:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49a0f expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.728952:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.728971:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49a0f expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.729024:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.729039:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 9 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.729055:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346127744:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826283.729487:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3bc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.739486:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826283.739518:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346127744:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826283.739559:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a0f expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.739576:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:2.0:1787826283.739579:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826283.739583:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a0f expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.739593:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:2.0:1787826283.739660:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35098:x1874663346127616:12345-0@lo:101:mdt00_013.0 Request processed in 13596us (13710us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.739668:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332535808 00000100:00100000:2.0:1787826283.739670:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23348 00000100:00100000:0.0:1787826283.739676:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332535808 00010000:00010000:0.0:1787826283.739712:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 10 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826283.739750:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21861 00000100:00100000:0.0:1787826283.739761:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332535808:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826283.739778:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826283.739785:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 10 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.739799:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 10 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.739823:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f499bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801012 expref: 10 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.739837:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:0.0:1787826283.739859:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a0f expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.739869:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49a0f expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.739878:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49a0f expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.739909:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a0f expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826283.739923:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346127872:0@lo:105: 00000100:00100000:0.0:1787826283.739959:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826283.740004:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.740149:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e762a00 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346127616:0@lo:101:mdt00_013.0 00010000:00010000:3.0:1787826283.740165:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.740175:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.742321:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.742371:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.742406:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.742440:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.742472:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.742518:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.742561:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332535936 00000100:00100000:3.0:1787826283.742570:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332535936 00000100:00100000:3.0:1787826283.742754:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2210 00000100:00100000:3.0:1787826283.742771:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847803050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332535936:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826283.742847:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847803050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332535936:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 72us (467us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.742858:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2210 00000100:00100000:3.0:1787826283.742903:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332536064 00000100:00100000:3.0:1787826283.742908:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332536064 00000100:00100000:3.0:1787826283.742959:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332536192 00000100:00100000:3.0:1787826283.742963:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332536192 00000100:00100000:3.0:1787826283.742992:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332536576 00000100:00100000:3.0:1787826283.742995:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332536576 00000100:00100000:3.0:1787826283.743030:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8770 00000100:00100000:3.0:1787826283.743040:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332536064:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826283.743076:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332536064:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 37us (764us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.743084:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8770 00000100:00100000:3.0:1787826283.743130:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8771 00000100:00100000:3.0:1787826283.743141:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332536192:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826283.743172:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332536192:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 33us (765us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.743179:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8771 00000100:00100000:3.0:1787826283.743200:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8772 00000100:00100000:3.0:1787826283.743210:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332536576:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826283.743240:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332536576:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 32us (724us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.743248:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8772 00000100:00100000:1.0:1787826283.745309:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346127872 00000100:00100000:1.0:1787826283.745323:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346127872 00000100:00100000:1.0:1787826283.745399:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7816 00000100:00100000:1.0:1787826283.745409:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346127872:12345-0@lo:105: 00010000:00010000:1.0:1787826283.745426:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.745437:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.745459:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.745468:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.745476:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.745485:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.745521:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69600) 00000100:00100000:1.0:1787826283.745593:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346127872:12345-0@lo:105: Request processed in 186us (5632us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826283.745602:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7816 00010000:00010000:3.0:1787826283.745609:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.745621:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826283.745632:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332536448 00000100:00100000:1.0:1787826283.745637:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332536448 00000004:00080000:3.0:1787826283.745662:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18d:0x0] 00000100:00100000:1.0:1787826283.745666:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332536320 00000100:00100000:1.0:1787826283.745671:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332536320 00000100:00100000:1.0:1787826283.745698:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14241 00010000:00010000:3.0:1787826283.745706:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826283.745707:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332536448:12345-192.168.203.56@tcp:400:kworker.0 00010000:00010000:3.0:1787826283.745720:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.745729:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.745750:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128000:0@lo:101:mdt00_013.0 00000100:00100000:1.0:1787826283.745769:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332536448:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 62us (3329us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826283.745776:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14241 00000100:00100000:1.0:1787826283.745794:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14242 00000100:00100000:3.0:1787826283.745795:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.745804:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332536320:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826283.745823:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3bc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826283.745849:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332536320:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 36us (3367us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826283.745858:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14242 00000100:00080000:1.0:1787826283.745886:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826283.745891:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826283.745963:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346128000 00000100:00100000:0.0:1787826283.745969:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346128000 00000100:00100000:0.0:1787826283.746009:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23349 00000100:00100000:0.0:1787826283.746016:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:35098:x1874663346128000:12345-0@lo:101:mdt00_013.0 00010000:00010000:0.0:1787826283.746025:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826283.746043:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f49a24 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49a1d expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.746064:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f49a24 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49a1d expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.746077:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f49a24 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a1d expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.746090:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f794b57a700) 00010000:00010000:0.0:1787826283.746125:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826283.746130:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826283.746172:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346128000:12345-0@lo:101:mdt00_013.0 Request processed in 157us (381us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.746178:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23349 00000100:00100000:0.0:1787826283.746214:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346127872:0@lo:105: 00010000:00010000:0.0:1787826283.746230:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826283.746259:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332535808:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 6500us (6782us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.746267:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21861 00000100:00100000:1.0:1787826283.747138:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128000:0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826283.747157:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a24 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.747170:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a24 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826283.747187:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18d:0x0] 00000100:00100000:1.0:1787826283.747231:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128128:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826283.747271:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.747303:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b4dfcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.747370:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346128128 00000100:00100000:3.0:1787826283.747376:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346128128 00000100:00100000:3.0:1787826283.747421:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23350 00000100:00100000:3.0:1787826283.747433:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346128128:12345-0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826283.747520:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346128128:12345-0@lo:1000:mdt00_013.0 Request processed in 90us (248us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.747528:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23350 00000100:00100000:1.0:1787826283.747798:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128128:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826283.747858:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128256:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826283.747889:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.747913:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b4df0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826283.748021:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346128256 00000100:00100000:0.0:1787826283.748027:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346128256 00000100:00100000:0.0:1787826283.748061:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23351 00000100:00100000:0.0:1787826283.748069:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346128256:12345-0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826283.748195:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346128256:12345-0@lo:1000:mdt00_013.0 Request processed in 126us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.748204:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23351 00000100:00100000:1.0:1787826283.748250:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128256:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826283.748291:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128384:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826283.748320:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.748342:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b4dfcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826283.748450:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346128384 00000100:00100000:0.0:1787826283.748454:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346128384 00000100:00100000:0.0:1787826283.748488:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23352 00000100:00100000:0.0:1787826283.748607:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346128384:12345-0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826283.748758:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346128384:12345-0@lo:1000:mdt00_013.0 Request processed in 262us (437us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826283.748767:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23352 00000100:00100000:1.0:1787826283.748793:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304700 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128384:0@lo:1000:mdt00_013.0 00010000:00010000:1.0:1787826283.748859:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.748869:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.748879:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826283.748892:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128512:0@lo:101:mdt00_013.0 00000100:00100000:1.0:1787826283.748917:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.748941:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f796cce2f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826283.749247:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346128512 00000100:00100000:0.0:1787826283.749253:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346128512 00000100:00100000:0.0:1787826283.749288:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23353 00000100:00100000:0.0:1787826283.749296:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346128512:12345-0@lo:101:mdt00_013.0 00010000:00010000:0.0:1787826283.749305:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826283.749321:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49a32 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49a2b expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.749342:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49a32 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49a2b expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.749355:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49a32 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a2b expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.749480:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:0.0:1787826283.749483:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826283.749487:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:0.0:1787826283.749601:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35098:x1874663346128512:12345-0@lo:101:mdt00_013.0 Request processed in 305us (682us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.749607:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23353 00000100:00100000:1.0:1787826283.749881:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346128512:0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826283.749898:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.749911:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.749948:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.749957:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.749966:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787826283.752942:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].224, 64 off22528 00000004:00080000:0.0:1787826283.752960:0:35098:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8095964853725ns, next at 8096964850367ns, rc = 0 00000040:00080000:0.0:1787826283.752992:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1179, 384 off1072088 00000040:00080000:0.0:1787826283.753053:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1498, 464 off1512080 00000004:00080000:0.0:1787826283.753059:0:35098:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f796cce2f00 with version 6789 00000004:00080000:1.0:1787826283.753771:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6789 rpc_version 6789 00000020:00100000:1.0:1787826283.753807:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826283.753818:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78435b2d80 x1874663346128768/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826283.753853:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346128768:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826283.753925:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.754006:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.757540:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346128768 00000100:00100000:3.0:1787826283.757556:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346128768 00000100:00100000:3.0:1787826283.757629:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23354 00000100:00100000:3.0:1787826283.757639:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8c050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346128768:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826283.758283:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8c050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346128768:12345-0@lo:1000:osp_up0-1.0 Request processed in 644us (4356us total) trans 8589957325 rc 0/0 00000100:00100000:3.0:1787826283.758292:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23354 00000100:00100000:1.0:1787826283.758596:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346128768:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826283.758745:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758763:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758773:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758781:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758787:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758793:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758819:0:35098:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826283.758848:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758856:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758862:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758867:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826283.758872:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758879:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758891:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:0.0:1787826283.758894:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826283.758899:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758905:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f49a39 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758918:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758926:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758932:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758952:0:35098:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787826283.758966:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18d:0x0] 00010000:00010000:0.0:1787826283.758973:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758980:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.758986:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.759008:0:35098:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787826283.759012:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18d:0x0] 00010000:00010000:0.0:1787826283.759017:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a24 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.759029:0:35098:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a24 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759093:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1787826283.759100:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759136:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826283.759143:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826283.759144:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223832:x1874663332535680:12345-192.168.203.56@tcp:36:rm.0 Request processed in 33401us (34030us total) trans 4294985342 rc 0/0 00000100:00100000:0.0:1787826283.759160:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68257 00010000:00010000:2.0:1787826283.759160:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759175:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:2.0:1787826283.759179:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826283.759182:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759188:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f49a40 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e43:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759203:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826283.759208:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759215:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826283.759219:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826283.759228:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.759245:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826283.759250:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.759256:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826283.759259:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.759261:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759266:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000004:00010000:1.0:1787826283.759268:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759287:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.759291:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759296:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f49a2b lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a32 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.759298:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826283.759303:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346128896:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826283.759311:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826283.759313:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 1/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.759314:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826283.759316:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826283.759321:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f49a0f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a16 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.759334:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826283.759336:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826283.759339:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826283.759346:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.759347:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826283.759359:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346129024:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826283.759867:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826283.759930:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826283.760039:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346128896 00000100:00100000:0.0:1787826283.760046:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346129024 00000100:00100000:2.0:1787826283.760047:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346128896 00000100:00100000:0.0:1787826283.760053:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346129024 00000100:00100000:0.0:1787826283.760135:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11011 00000100:00100000:0.0:1787826283.760145:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346129024:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826283.760156:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826283.760174:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f49a16 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49a0f expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.760189:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:0.0:1787826283.760197:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826283.760238:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346129024:12345-0@lo:103:ldlm_bl.0 Request processed in 95us (372us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.760244:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11011 00000100:00100000:3.0:1787826283.760304:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346129024:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826283.760890:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11012 00000100:00100000:0.0:1787826283.760905:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346128896:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826283.760914:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826283.760928:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49a32 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e43:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49a2b expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.760942:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e43:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:0.0:1787826283.760950:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826283.762014:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346128896:12345-0@lo:103:ldlm_bl.0 Request processed in 1108us (2663us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.762025:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11012 00000100:00100000:3.0:1787826283.762347:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346128896:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826283.765730:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826283.765770:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826283.765877:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332536704 00000100:00100000:0.0:1787826283.765887:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332536704 00000100:00100000:0.0:1787826283.765963:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11013 00000100:00100000:0.0:1787826283.765975:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332536704:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826283.765989:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826283.766024:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4985d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa9 expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.766045:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826283.766050:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826283.766090:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332536960 00000100:00100000:2.0:1787826283.766094:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332536960 00000100:00100000:0.0:1787826283.766226:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332536704:12345-192.168.203.56@tcp:103:rm.0 Request processed in 250us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.766236:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11013 00000100:00100000:2.0:1787826283.766240:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9309 00000100:00100000:2.0:1787826283.766252:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:223832:x1874663332536960:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:2.0:1787826283.766431:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:223832:x1874663332536960:12345-192.168.203.56@tcp:35:rm.0 Request processed in 181us (714us total) trans 4294985343 rc 0/0 00000100:00100000:2.0:1787826283.766440:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9309 00000100:00100000:0.0:1787826283.770475:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826283.770740:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332537216 00000100:00100000:1.0:1787826283.770750:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332537216 00000100:00100000:1.0:1787826283.770808:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68258 00000100:00100000:1.0:1787826283.770818:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:223832:x1874663332537216:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826283.770882:0:24702:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826283.770910:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f499d7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e42:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801020 expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.770931:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4999f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801004 expref: 13 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.770945:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4986b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fb0 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.770956:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:1.0:1787826283.770960:0:24702:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826283.771005:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.771013:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.771022:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826283.771046:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346129152:0@lo:101:mdt00_006.0 00000100:00100000:1.0:1787826283.771100:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.771765:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.771819:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346129152 00000100:00100000:3.0:1787826283.771825:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346129152 00000100:00100000:3.0:1787826283.771876:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23355 00000100:00100000:3.0:1787826283.771984:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346129152:12345-0@lo:101:mdt00_006.0 00010000:00010000:3.0:1787826283.772003:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826283.772019:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49a47 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.772043:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.772052:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.772061:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.772085:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.772739:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 7 pid: 35093 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.772758:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189ed80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346129280:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826283.772860:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.772869:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 7 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.772879:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346129408:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826283.772904:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3b480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.783407:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189ed80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346129280:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826283.789368:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826283.789439:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332537472 00000100:00100000:2.0:1787826283.789449:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332537472 00010000:00010000:2.0:1787826283.789490:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 8 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826283.789541:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21862 00000100:00100000:2.0:1787826283.789552:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843435c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332537472:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826283.789566:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826283.789574:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 8 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.789586:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 8 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.789609:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826283.789616:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.789627:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826283.789655:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826283.789679:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843435c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332537472:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 129us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826283.789687:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21862 00010000:00010000:0.0:1787826283.789798:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49841 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fa2 expref: 7 pid: 6527 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826283.789816:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346129408:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826283.790248:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.790264:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826283.790267:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826283.790271:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.790279:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68800, rc 0) 00000100:00100000:0.0:1787826283.790329:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:24702:x1874663346129152:12345-0@lo:101:mdt00_006.0 Request processed in 18444us (19228us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.790337:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23355 00000100:00100000:0.0:1787826283.790376:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346129152:0@lo:101:mdt00_006.0 00010000:00010000:0.0:1787826283.790391:0:24702:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.790403:0:24702:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826283.791634:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332537344 00000100:00100000:2.0:1787826283.791643:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332537344 00010000:00010000:2.0:1787826283.791676:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 8 pid: 35093 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.792271:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21863 00000100:00100000:3.0:1787826283.792287:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332537344:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826283.792300:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826283.792307:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 8 pid: 35093 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.792321:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 8 pid: 35093 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.792340:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f498b1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fcc expref: 8 pid: 35093 timeout: 8195 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.792362:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826283.792368:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.792379:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.792388:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.792420:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.792434:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346129536:0@lo:105: 00000100:00100000:3.0:1787826283.792486:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826283.794573:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3bc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826283.800985:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346129536 00000100:00100000:1.0:1787826283.801003:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346129536 00000100:00100000:0.0:1787826283.801782:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7817 00000100:00100000:0.0:1787826283.801797:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346129536:12345-0@lo:105: 00010000:00010000:0.0:1787826283.801812:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.801823:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.801831:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.801854:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.801862:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.801870:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.801901:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf200) 00000100:00100000:0.0:1787826283.801986:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346129536:12345-0@lo:105: Request processed in 193us (9499us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.801994:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7817 00010000:00010000:2.0:1787826283.802051:0:24702:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.802064:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826283.802092:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826283.802162:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.802172:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.802184:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826283.802204:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642300 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346129664:0@lo:101:mdt00_006.0 00000100:00100000:2.0:1787826283.802242:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826283.802458:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346129536:0@lo:105: 00000100:00100000:2.0:1787826283.802465:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00010000:00010000:3.0:1787826283.802475:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826283.802516:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346129664 00000100:00100000:0.0:1787826283.802522:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346129664 00000100:00100000:3.0:1787826283.802570:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332537344:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 10286us (12911us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.802588:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21863 00000100:00100000:0.0:1787826283.802595:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23356 00000100:00100000:0.0:1787826283.802602:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:24702:x1874663346129664:12345-0@lo:101:mdt00_006.0 00010000:00010000:0.0:1787826283.802618:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826283.802634:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49a5c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49a55 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.802657:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49a5c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49a55 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.802671:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49a5c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a55 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.802684:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a600) 00010000:00010000:0.0:1787826283.802687:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826283.802690:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826283.802736:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346129664:12345-0@lo:101:mdt00_006.0 Request processed in 134us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.802742:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23356 00000100:00100000:3.0:1787826283.802800:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642300 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346129664:0@lo:101:mdt00_006.0 00010000:00010000:3.0:1787826283.802814:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.802829:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826283.802853:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826283.802886:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346129792:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826283.802915:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826283.808725:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346129792 00000100:00100000:2.0:1787826283.808739:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346129792 00000100:00100000:2.0:1787826283.808807:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23357 00000100:00100000:2.0:1787826283.808816:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346129792:12345-0@lo:1000:mdt00_006.0 00000100:00100000:2.0:1787826283.809030:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346129792:12345-0@lo:1000:mdt00_006.0 Request processed in 214us (6113us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826283.809038:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23357 00000100:00100000:3.0:1787826283.810002:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3b480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.810033:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346129792:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826283.810102:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346129920:0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826283.811972:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826283.812064:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7967909f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826283.812254:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346129920 00000100:00100000:0.0:1787826283.812264:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346129920 00000100:00100000:0.0:1787826283.812366:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23358 00000100:00100000:0.0:1787826283.812376:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346129920:12345-0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826283.812544:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346129920:12345-0@lo:1000:mdt00_006.0 Request processed in 168us (574us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826283.812551:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23358 00000100:00100000:3.0:1787826283.812688:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346129920:0@lo:1000:mdt00_006.0 00010000:00010000:3.0:1787826283.812759:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.812769:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.812779:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.812796:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346130048:0@lo:101:mdt00_006.0 00000100:00100000:3.0:1787826283.812823:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826283.812857:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3bc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826283.812982:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346130048 00000100:00100000:2.0:1787826283.812989:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346130048 00000100:00100000:2.0:1787826283.813028:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23359 00000100:00100000:2.0:1787826283.813036:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346130048:12345-0@lo:101:mdt00_006.0 00010000:00010000:2.0:1787826283.813045:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826283.813061:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49a63 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.813084:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 6 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.813093:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49a63 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.813154:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 6 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.813167:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 6 pid: 6527 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826283.813178:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346130176:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826283.813241:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826283.818255:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826283.818800:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346130176:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826283.818850:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a63 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.818863:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:2.0:1787826283.818866:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826283.818871:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a63 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.818882:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826283.818951:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:24702:x1874663346130048:12345-0@lo:101:mdt00_006.0 Request processed in 5916us (6126us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826283.818960:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23359 00000100:00100000:0.0:1787826283.819417:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332537600 00000100:00100000:0.0:1787826283.819425:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332537600 00010000:00010000:0.0:1787826283.819460:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 7 pid: 6527 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826283.819497:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21864 00000100:00100000:0.0:1787826283.819506:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c82850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332537600:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826283.819520:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826283.819526:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 7 pid: 6527 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.819539:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 7 pid: 6527 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.819560:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f498cd lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760800fd3 expref: 7 pid: 6527 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.819575:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826283.819580:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a63 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.819590:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49a63 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.819598:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49a63 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.819627:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a63 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826283.819642:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346130304:0@lo:105: 00000100:00100000:0.0:1787826283.819684:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826283.819726:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826283.819759:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826283.819772:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346130304 00000100:00100000:1.0:1787826283.819776:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346130304 00000100:00100000:1.0:1787826283.819817:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7818 00000100:00100000:1.0:1787826283.819825:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78557c2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346130304:12345-0@lo:105: 00010000:00010000:1.0:1787826283.819836:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.819857:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.819866:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.819874:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.819882:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.819891:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.819904:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68200) 00000100:00100000:1.0:1787826283.819938:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78557c2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346130304:12345-0@lo:105: Request processed in 113us (252us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826283.819944:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7818 00000100:00100000:0.0:1787826283.820046:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346130304:0@lo:105: 00010000:00010000:0.0:1787826283.820062:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826283.820145:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c82850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332537600:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 600us (1861us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.820153:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21864 00000100:00100000:3.0:1787826283.829082:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346130048:0@lo:101:mdt00_006.0 00010000:00010000:3.0:1787826283.829150:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.829266:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.829282:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.829295:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.829319:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.829331:0:24702:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.829339:0:24702:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.829376:0:24702:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.829392:0:24702:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 11 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826283.829415:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001_UUID:24702:1874663346130432:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826283.829517:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826283.836907:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826283.837075:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332537728 00000100:00100000:0.0:1787826283.837084:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332537728 00010000:00010000:0.0:1787826283.837168:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 12 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826283.837209:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21865 00000100:00100000:0.0:1787826283.837219:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332537728:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826283.837233:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826283.837240:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 12 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.837253:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 12 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.837274:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:0.0:1787826283.837279:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.837289:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.837297:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.837316:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826283.837366:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332537728:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 148us (466us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.837373:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21865 00010000:00010000:2.0:1787826283.840565:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f499ad lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080100b expref: 11 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826283.840602:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001_UUID:24702:1874663346130432:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826283.840630:0:24702:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.840641:0:24702:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.840648:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.840705:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.840714:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.840735:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826283.840757:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346130560:0@lo:101:mdt00_006.0 00000100:00100000:2.0:1787826283.840813:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826283.840860:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826283.845150:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346130560 00000100:00100000:1.0:1787826283.845166:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346130560 00000100:00100000:1.0:1787826283.845246:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23360 00000100:00100000:1.0:1787826283.845256:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:24702:x1874663346130560:12345-0@lo:101:mdt00_006.0 00010000:00010000:1.0:1787826283.845278:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826283.845307:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a86 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49a7f expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.845338:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a86 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49a7f expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.845352:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a86 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a7f expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.845361:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:1.0:1787826283.845364:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826283.845368:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826283.845445:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:24702:x1874663346130560:12345-0@lo:101:mdt00_006.0 Request processed in 191us (4631us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826283.845452:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23360 00000100:00100000:2.0:1787826283.845590:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346130560:0@lo:101:mdt00_006.0 00010000:00010000:2.0:1787826283.845610:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826283.845621:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826283.846331:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346130688:0@lo:403:mdt00_006.0 00000100:00100000:2.0:1787826283.846390:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826283.846414:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.846543:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346130688 00000100:00100000:3.0:1787826283.846551:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346130688 00000100:00100000:3.0:1787826283.846588:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23361 00000100:00100000:3.0:1787826283.846597:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:24702:x1874663346130688:12345-0@lo:403:mdt00_006.0 00000100:00100000:3.0:1787826283.848417:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:24702:x1874663346130688:12345-0@lo:403:mdt00_006.0 Request processed in 1815us (2020us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.848429:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23361 00000100:00100000:0.0:1787826283.849026:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346130688:0@lo:403:mdt00_006.0 00000040:00080000:0.0:1787826283.852270:0:24702:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1180, 952 off1073040 00000040:00080000:0.0:1787826283.852448:0:24702:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1499, 1032 off1513112 00000004:00080000:0.0:1787826283.852454:0:24702:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784548a9c0 with version 6790 00000004:00080000:1.0:1787826283.852866:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6790 rpc_version 6790 00000020:00100000:1.0:1787826283.852900:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826283.852911:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7843434e00 x1874663346130944/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826283.852933:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346130944:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826283.852995:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826283.853028:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826283.853099:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346130944 00000100:00100000:3.0:1787826283.853124:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346130944 00000100:00100000:3.0:1787826283.853179:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23362 00000100:00100000:3.0:1787826283.853189:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346130944:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826283.853510:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346130944:12345-0@lo:1000:osp_up0-1.0 Request processed in 322us (513us total) trans 8589957326 rc 0/0 00000100:00100000:3.0:1787826283.853518:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23362 00000100:00100000:1.0:1787826283.853620:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346130944:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787826283.855339:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e42:0x0]: returning size 4096 00010000:00010000:3.0:1787826283.855371:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855381:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855388:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855394:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826283.855399:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855409:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855434:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x18f:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:3.0:1787826283.855437:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826283.855440:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855445:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f49a78 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855464:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855469:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855474:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855543:0:24702:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826283.855550:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18d:0x0] 00010000:00010000:3.0:1787826283.855556:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855560:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855564:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855570:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826283.855572:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855576:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855581:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:3.0:1787826283.855582:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826283.855584:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855587:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49a71 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855593:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855598:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.855602:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.856747:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826283.856758:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.856768:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826283.856773:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826283.856782:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.856833:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.856851:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826283.856887:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 1/0,0 mode: --/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.856894:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f49a7f lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a86 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826283.856905:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346131072:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826283.856914:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826283.856917:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.856923:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826283.856926:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826283.856932:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.856947:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.856951:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826283.856969:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826283.856971:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826283.856977:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826283.857023:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346131072 00000100:00100000:1.0:1787826283.857030:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346131072 00000100:00100000:1.0:1787826283.857077:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11014 00000100:00100000:1.0:1787826283.857086:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+15:3655:x1874663346131072:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826283.857099:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826283.857138:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a86 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49a7f expref: 15 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826283.857151:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:1.0:1787826283.857159:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826283.857192:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+14:3655:x1874663346131072:12345-0@lo:103:ldlm_bl.0 Request processed in 107us (221us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826283.857199:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11014 00000100:00100000:1.0:1787826283.857237:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346131072:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826283.858227:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346131200:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826283.858272:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826283.858538:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826283.858544:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826283.858946:0:24702:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1787826283.858957:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68200/0xe21309aa81f49a63 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a6a expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826283.858996:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826283.859003:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.859013:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.859021:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.859056:0:24702:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826283.859062:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826283.859068:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.859075:0:24702:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.859089:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826283.859095:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.859102:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826283.859123:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826283.859131:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.859161:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826283.859162:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:223832:x1874663332537216:12345-192.168.203.56@tcp:36:rm.0 Request processed in 88343us (88694us total) trans 4294985344 rc 0/0 00010000:00010000:0.0:1787826283.859167:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826283.859170:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68258 00010000:00010000:0.0:1787826283.859182:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.859188:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f49a47 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a4e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826283.859199:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826283.859200:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826283.859202:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826283.859211:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346131328:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826283.859244:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826283.859247:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826283.861916:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346131200 00000100:00100000:2.0:1787826283.861930:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346131200 00000100:00100000:2.0:1787826283.861999:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346131328 00000100:00100000:2.0:1787826283.862002:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346131328 00000100:00100000:3.0:1787826283.862290:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11015 00000100:00100000:3.0:1787826283.862303:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843435880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346131200:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826283.862318:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826283.862343:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49a6a lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49a63 expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.862363:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:3.0:1787826283.862375:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826283.862453:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843435880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+14:3652:x1874663346131200:12345-0@lo:103:ldlm_bl.0 Request processed in 154us (4179us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.862460:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11015 00000100:00100000:3.0:1787826283.862479:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11016 00000100:00100000:3.0:1787826283.862486:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346131328:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826283.862494:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826283.862505:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49a4e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49a47 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826283.862516:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826283.862519:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826283.862545:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346131328:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (3300us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826283.862551:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11016 00000100:00100000:1.0:1787826283.862765:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346131200:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826283.863952:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346131328:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826283.974543:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826283.977680:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332537856 00000100:00100000:3.0:1787826283.977694:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332537856 00000100:00100000:0.0:1787826283.978887:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68259 00000100:00100000:0.0:1787826283.978909:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843435c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223840:x1874663332537856:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:0.0:1787826283.978969:0:35092:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345618944 00000100:00100000:0.0:1787826283.979055:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843435c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223840:x1874663332537856:12345-192.168.203.56@tcp:41:stat.0 Request processed in 155us (4520us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826283.979062:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68259 00000100:00100000:0.0:1787826284.032616:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826284.032719:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332537984 00000100:00100000:1.0:1787826284.032729:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332537984 00000100:00100000:1.0:1787826284.032809:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68260 00000100:00100000:1.0:1787826284.032821:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845affb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223844:x1874663332537984:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826284.032851:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826284.032875:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a8d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801035 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.032940:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.032958:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.032968:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826284.033014:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826284.033018:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826284.033038:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.033064:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a8d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.033085:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.033094:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826284.033097:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826284.033101:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826284.033351:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845affb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223844:x1874663332537984:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 532us (837us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.033358:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68260 00000100:00080000:2.0:1787826284.033862:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826284.037101:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826284.037453:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332538112 00000100:00100000:1.0:1787826284.037463:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332538112 00000100:00100000:1.0:1787826284.037651:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68261 00000100:00100000:1.0:1787826284.037663:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223844:x1874663332538112:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826284.037684:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826284.037705:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a9b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080103c expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.037773:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.037786:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.037798:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.037809:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.037819:0:11629:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49a5c lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49a55 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.037827:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.037872:0:11629:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49a5c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49a55 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826284.037890:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346131456:0@lo:104: 00000100:00100000:1.0:1787826284.037943:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826284.038030:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826284.042287:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346131456 00000100:00100000:0.0:1787826284.042301:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346131456 00000100:00100000:0.0:1787826284.042359:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7819 00000100:00100000:0.0:1787826284.042369:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346131456:12345-0@lo:104: 00010000:00010000:0.0:1787826284.042387:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.043132:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826284.043141:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346131456:12345-0@lo:104: Request processed in 773us (5196us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.043150:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7819 00010000:00010000:1.0:1787826284.043320:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826284.043328:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.043343:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826284.043349:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826284.043358:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.043396:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.043402:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826284.043430:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.043438:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f49a55 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49a5c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.043459:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826284.043461:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826284.043463:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826284.043470:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826284.043490:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346131584:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826284.043555:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826284.043593:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346131584 00000100:00100000:1.0:1787826284.043599:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346131584 00010000:00010000:1.0:1787826284.043624:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49a5c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49a55 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826284.043654:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21866 00000100:00100000:1.0:1787826284.043663:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346131584:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826284.043673:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826284.044006:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346131456:0@lo:104: 00010000:00010000:3.0:1787826284.044031:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826284.048978:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957326 is committed 00010000:00010000:2.0:1787826284.049167:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49a5c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49a55 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.049199:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a600) 00010000:00010000:2.0:1787826284.049204:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.049215:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.049222:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.049256:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826284.049336:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346131584:12345-0@lo:103:ldlm_bl.0 Request processed in 5672us (5779us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.049349:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21866 00010000:00010000:3.0:1787826284.049532:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049541:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049602:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049631:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049638:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049646:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826284.049651:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346131584:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826284.049653:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826284.049662:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049671:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049683:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826284.049687:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826284.049691:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049697:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49aa2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049709:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049715:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049720:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049726:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826284.049729:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049735:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049743:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a600) 00010000:00010000:3.0:1787826284.049745:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826284.049748:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049754:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49aa9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049796:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a9b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080103c expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049806:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826284.049808:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826284.049812:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49a9b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080103c expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.049821:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:3.0:1787826284.049889:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223844:x1874663332538112:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 12228us (12797us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826284.049896:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68261 00000100:00100000:1.0:1787826284.058855:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826284.059723:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332538240 00000100:00100000:0.0:1787826284.059734:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332538240 00000100:00100000:0.0:1787826284.059797:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68262 00000100:00100000:0.0:1787826284.059808:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:223844:x1874663332538240:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826284.059829:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826284.059861:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ab0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801043 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.059928:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.059942:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.059954:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826284.060001:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826284.060006:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826284.060025:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.060049:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ab0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.060068:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.060083:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826284.060087:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826284.060091:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826284.060173:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:223844:x1874663332538240:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 367us (1337us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.060181:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68262 00000100:00100000:0.0:1787826284.067471:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826284.067558:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332538496 00000100:00100000:1.0:1787826284.067567:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332538496 00000100:00100000:1.0:1787826284.067658:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68263 00000100:00100000:1.0:1787826284.067669:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223844:x1874663332538496:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:1.0:1787826284.067780:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346131712:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826284.067830:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826284.067874:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826284.068033:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346131712 00000100:00100000:0.0:1787826284.068038:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346131712 00000100:00100000:0.0:1787826284.068075:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23363 00000100:00100000:0.0:1787826284.068083:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346131712:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826284.068172:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346131712:12345-0@lo:1000:mdt00_012.0 Request processed in 89us (341us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.068179:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23363 00000100:00080000:2.0:1787826284.068253:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826284.068353:0:35095:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957326 gen 2 00000004:00080000:1.0:1787826284.068359:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957324 committed transno 8589957326 00000004:00080000:1.0:1787826284.068378:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957325 committed transno 8589957326 00000004:00080000:1.0:1787826284.068391:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957326 committed transno 8589957326 00000100:00100000:1.0:1787826284.068408:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346131712:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826284.068448:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346131840:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826284.068473:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826284.068495:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826284.068510:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346131840 00000100:00100000:1.0:1787826284.068513:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346131840 00000100:00100000:1.0:1787826284.068546:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23364 00000100:00100000:1.0:1787826284.068553:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346131840:12345-0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826284.068784:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346131840:12345-0@lo:1000:mdt00_012.0 Request processed in 230us (308us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826284.068792:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23364 00000100:00100000:2.0:1787826284.068874:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346131840:0@lo:1000:mdt00_012.0 00010000:00010000:2.0:1787826284.068932:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.068942:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.068952:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826284.068965:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346131968:0@lo:101:mdt00_012.0 00000100:00100000:2.0:1787826284.068991:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.069015:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826284.069142:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346131968 00000100:00100000:0.0:1787826284.069147:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346131968 00000100:00100000:0.0:1787826284.069169:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23365 00000100:00100000:0.0:1787826284.069175:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346131968:12345-0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826284.069184:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826284.069198:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49abe expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.069219:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.069227:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.069236:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49abe expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.069255:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.069267:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 7 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826284.069276:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346132096:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826284.069346:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.069355:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 6 pid: 35080 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826284.069363:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346132224:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826284.069380:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826284.072462:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826284.072546:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826284.072660:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332538624 00000100:00100000:1.0:1787826284.072673:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332538624 00010000:00010000:1.0:1787826284.072721:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 8 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826284.072769:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332538752 00000100:00100000:1.0:1787826284.072774:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332538752 00010000:00010000:1.0:1787826284.072788:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 7 pid: 35080 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826284.072819:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21867 00000100:00100000:1.0:1787826284.072827:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332538624:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826284.072852:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826284.072859:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 8 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.072872:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 8 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.072892:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826284.072898:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49abe expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.072907:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826284.072956:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332538624:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 130us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.072962:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21867 00000100:00100000:1.0:1787826284.072978:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21868 00000100:00100000:1.0:1787826284.072984:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c83450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332538752:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826284.072990:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826284.072994:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 7 pid: 35080 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073003:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 7 pid: 35080 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073014:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826284.073019:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49abe expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073027:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49abe expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073035:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49abe expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073063:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49abe expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826284.073076:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346132352:0@lo:105: 00000100:00100000:1.0:1787826284.073171:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826284.073208:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826284.073381:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346132352 00000100:00100000:1.0:1787826284.073385:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346132352 00000100:00100000:1.0:1787826284.073407:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7820 00000100:00100000:1.0:1787826284.073413:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346132352:12345-0@lo:105: 00010000:00010000:1.0:1787826284.073422:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073432:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073440:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073447:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073455:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073465:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.073479:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef000) 00000100:00100000:1.0:1787826284.073854:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346132352:12345-0@lo:105: Request processed in 438us (680us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.073861:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7820 00000100:00100000:2.0:1787826284.074300:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346132352:0@lo:105: 00010000:00010000:2.0:1787826284.074320:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826284.074356:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c83450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332538752:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1370us (1808us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.074368:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21868 00010000:00010000:0.0:1787826284.075402:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49ab7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801043 expref: 7 pid: 6529 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826284.075435:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346132096:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826284.075463:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f49a94 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801035 expref: 6 pid: 35080 timeout: 8196 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826284.075483:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346132224:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826284.075749:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49abe expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.075761:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826284.075765:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826284.075769:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826284.075874:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346131968:12345-0@lo:101:mdt00_012.0 Request processed in 6697us (6880us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.075883:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23365 00000100:00100000:2.0:1787826284.076429:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346131968:0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826284.076454:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826284.076485:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826284.076670:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.076680:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.076690:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826284.076715:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132480:0@lo:101:mdt00_012.0 00000100:00100000:2.0:1787826284.076767:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.076809:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826284.076903:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346132480 00000100:00100000:3.0:1787826284.076913:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346132480 00000100:00100000:3.0:1787826284.076968:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23366 00000100:00100000:3.0:1787826284.076980:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346132480:12345-0@lo:101:mdt00_012.0 00010000:00010000:3.0:1787826284.076999:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826284.077018:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49ad3 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49acc expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.077048:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49ad3 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49acc expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.077063:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49ad3 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49acc expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.077071:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961d81b00) 00010000:00010000:3.0:1787826284.077074:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826284.077077:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68800, rc 0) 00000100:00100000:3.0:1787826284.077307:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132480:12345-0@lo:101:mdt00_012.0 Request processed in 328us (539us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826284.077315:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23366 00000100:00100000:2.0:1787826284.077444:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132480:0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826284.077458:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.077468:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826284.077484:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826284.077598:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132608:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.077635:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.077701:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826284.078012:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346132608 00000100:00100000:1.0:1787826284.078020:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346132608 00000100:00100000:1.0:1787826284.078125:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23367 00000100:00100000:1.0:1787826284.078135:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132608:12345-0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826284.078223:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132608:12345-0@lo:1000:mdt00_012.0 Request processed in 89us (586us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.078231:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23367 00000100:00100000:2.0:1787826284.078293:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132608:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.078344:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132736:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.078386:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.078711:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b4dfa80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826284.078808:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346132736 00000100:00100000:0.0:1787826284.078816:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346132736 00000100:00100000:0.0:1787826284.078868:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23368 00000100:00100000:0.0:1787826284.078877:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132736:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826284.078988:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132736:12345-0@lo:1000:mdt00_012.0 Request processed in 112us (600us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826284.078996:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23368 00000100:00100000:2.0:1787826284.079065:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132736:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.079204:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132864:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.079233:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.079252:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b4dfa80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826284.079268:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346132864 00000100:00100000:2.0:1787826284.079272:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346132864 00000100:00100000:2.0:1787826284.079293:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23369 00000100:00100000:2.0:1787826284.079300:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132864:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.079588:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132864:12345-0@lo:1000:mdt00_012.0 Request processed in 287us (352us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826284.079596:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23369 00000100:00100000:2.0:1787826284.079627:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f685180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132864:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.079680:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132992:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.079706:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.079722:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826284.079735:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346132992 00000100:00100000:2.0:1787826284.079739:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346132992 00000100:00100000:2.0:1787826284.079757:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23370 00000100:00100000:2.0:1787826284.079763:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132992:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.079816:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346132992:12345-0@lo:1000:mdt00_012.0 Request processed in 52us (108us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.079823:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23370 00000100:00100000:2.0:1787826284.079861:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346132992:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.079909:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346133120:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.079932:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.079946:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b4dfa80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826284.079958:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346133120 00000100:00100000:2.0:1787826284.079961:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346133120 00000100:00100000:2.0:1787826284.079979:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23371 00000100:00100000:2.0:1787826284.079985:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346133120:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.080045:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346133120:12345-0@lo:1000:mdt00_012.0 Request processed in 59us (112us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.080051:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23371 00000100:00100000:2.0:1787826284.080075:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346133120:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.080150:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346133248:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.080175:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.080190:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826284.080201:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346133248 00000100:00100000:2.0:1787826284.080204:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346133248 00000100:00100000:2.0:1787826284.080221:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23372 00000100:00100000:2.0:1787826284.080226:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346133248:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826284.080267:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346133248:12345-0@lo:1000:mdt00_012.0 Request processed in 40us (91us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.080273:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23372 00000100:00100000:2.0:1787826284.080296:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346133248:0@lo:1000:mdt00_012.0 00000040:00080000:2.0:1787826284.085803:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1181, 2168 off1075208 00000040:00080000:2.0:1787826284.085936:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1500, 2368 off1515480 00000004:00080000:2.0:1787826284.085943:0:35095:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784b4dfa80 with version 6791 00000004:00080000:1.0:1787826284.086134:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6791 rpc_version 6791 00000020:00100000:1.0:1787826284.086193:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826284.086205:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b753480 x1874663346133504/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826284.086234:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346133504:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826284.086323:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826284.086378:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826284.086577:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346133504 00000100:00100000:3.0:1787826284.086588:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346133504 00000100:00100000:3.0:1787826284.086659:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23373 00000100:00100000:3.0:1787826284.086671:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346133504:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826284.087701:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346133504:12345-0@lo:1000:osp_up0-1.0 Request processed in 1025us (1371us total) trans 8589957327 rc 0/0 00000100:00100000:3.0:1787826284.087713:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23373 00000100:00100000:3.0:1787826284.087825:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346133504:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826284.088077:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.088243:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.088262:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.088291:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.088300:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.088307:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.088348:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.088426:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.088440:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826284.088463:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346133632:0@lo:101:mdt00_012.0 00000100:00100000:2.0:1787826284.088514:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.088551:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826284.088619:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346133632 00000100:00100000:3.0:1787826284.088627:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346133632 00000100:00100000:3.0:1787826284.088679:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23374 00000100:00100000:3.0:1787826284.088691:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:35095:x1874663346133632:12345-0@lo:101:mdt00_012.0 00010000:00010000:3.0:1787826284.088713:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826284.088734:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49aef lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49ae8 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.088758:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49aef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49ae8 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.088771:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49aef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49ae8 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.088781:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:3.0:1787826284.088784:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826284.088788:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826284.088933:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346133632:12345-0@lo:101:mdt00_012.0 Request processed in 243us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826284.088940:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23374 00000100:00100000:1.0:1787826284.089061:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346133632:0@lo:101:mdt00_012.0 00010000:00010000:1.0:1787826284.089079:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089268:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826284.089302:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000090711ad1 reply state 0000000057484722 transno 4294985345 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089317:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089323:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:1.0:1787826284.089326:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826284.089329:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826284.089336:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089345:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826284.089425:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18e:0x0] 00000004:00010000:1.0:1787826284.089435:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000090711ad1 reply state 0000000057484722 transno 4294985345 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089445:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089451:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:1.0:1787826284.089453:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826284.089455:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787826284.089461:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e44:0x0]: returning size 4096 00000004:00080000:1.0:1787826284.089470:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826284.089481:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089488:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089495:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089557:0:35095:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826284.089562:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826284.089567:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.089575:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826284.089672:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223844:x1874663332538496:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 22003us (22208us total) trans 4294985345 rc 0/0 00000100:00100000:1.0:1787826284.089680:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68263 00010000:00010000:2.0:1787826284.093952:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826284.093972:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.093988:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826284.093997:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826284.094011:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.094071:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.094079:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826284.094315:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826284.094528:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332538880 00000100:00100000:3.0:1787826284.094538:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332538880 00000100:00100000:3.0:1787826284.094607:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68264 00000100:00100000:3.0:1787826284.094620:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223833:x1874663332538880:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826284.094640:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826284.094662:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49af6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080104a expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.094742:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.094758:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49abe expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.094766:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.094798:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49abe expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826284.094816:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346133888:0@lo:104: 00000100:00100000:3.0:1787826284.094889:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826284.094945:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7967909f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826284.095010:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346133888 00000100:00100000:0.0:1787826284.095016:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346133888 00000100:00100000:0.0:1787826284.095060:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7821 00000100:00100000:0.0:1787826284.095070:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346133888:12345-0@lo:104: 00010000:00010000:0.0:1787826284.095080:0:6509:0:(ldlm_lockd.c:2448:ldlm_callback_handler()) callback on lock 0xe21309aa81f49abe - lock disappeared 00010000:00010000:0.0:1787826284.095187:0:6509:0:(ldlm_lockd.c:2345:ldlm_callback_errmsg()) @@@ Operate with invalid parameter, NID=12345-0@lo lock=0xe21309aa81f49abe: rc = 0 req@ffff9f7843437800 x1874663346133888/t0(0) o104->lustre-MDT0001-mdtlov_UUID@0@lo:640/0 lens 328/224 e 0 to 0 dl 1787826295 ref 1 fl Interpret:/0/0 rc -22/-22 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826284.095200:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346133888:12345-0@lo:104: Request processed in 133us (312us total) trans 0 rc -22/-22 00000100:00100000:0.0:1787826284.095209:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7821 00010000:00010000:3.0:1787826284.095671:0:8752:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 0@lo) returned -22 from blocking AST (req@00000000ee01b065 x1874663346133888) - normal race ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49abe expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095702:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ac5 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49abe expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826284.095723:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346133888:0@lo:104: 00010000:00010000:3.0:1787826284.095745:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826284.095750:0:8752:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095763:0:8752:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095771:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095789:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095799:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095805:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826284.095875:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826284.095880:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826284.095908:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095938:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49af6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080104a expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095953:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080104a expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.095961:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826284.095963:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826284.095966:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:3.0:1787826284.096026:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223833:x1874663332538880:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1408us (1716us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826284.096034:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68264 00000100:00100000:3.0:1787826284.096582:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346133760:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826284.096653:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826284.096723:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.096734:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f49abe lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49ac5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.096754:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826284.096757:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826284.096759:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826284.097588:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346133760 00000100:00100000:0.0:1787826284.097600:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346133760 00000100:00100000:0.0:1787826284.097672:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11017 00000100:00100000:0.0:1787826284.097683:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346133760:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826284.097701:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826284.097704:0:6513:0:(ldlm_lockd.c:1754:ldlm_request_cancel()) ### server-side cancel handler stale lock (cookie e21309aa81f49ac5) 00010000:00010000:0.0:1787826284.097706:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826284.098753:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346133760:12345-0@lo:103:ldlm_bl.0 Request processed in 1067us (2094us total) trans 0 rc 116/116 00000100:00100000:0.0:1787826284.098765:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11017 00000100:00100000:2.0:1787826284.098904:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346133760:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826284.099757:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826284.099821:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332539008 00000100:00100000:2.0:1787826284.099829:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332539008 00000100:00100000:2.0:1787826284.099893:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68265 00000100:00100000:2.0:1787826284.099907:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843435880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223833:x1874663332539008:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826284.099934:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826284.099959:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49b04 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801051 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.100039:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.100061:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.100074:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.100086:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.100096:0:24703:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49ad3 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49acc expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.100145:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.100185:0:24703:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49ad3 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49acc expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826284.100202:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346134016:0@lo:104: 00000100:00100000:2.0:1787826284.100254:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.100303:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826284.101315:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346134016 00000100:00100000:1.0:1787826284.101328:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346134016 00000100:00100000:1.0:1787826284.101394:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7822 00000100:00100000:1.0:1787826284.101406:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346134016:12345-0@lo:104: 00010000:00010000:1.0:1787826284.101424:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.101527:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826284.101534:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346134016:12345-0@lo:104: Request processed in 130us (1280us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.101542:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7822 00000100:00100000:2.0:1787826284.101579:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346134016:0@lo:104: 00010000:00010000:2.0:1787826284.101613:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.101639:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826284.101648:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.101658:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826284.101664:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826284.101674:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.101711:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.101719:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826284.101755:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.101763:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49acc lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49ad3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826284.101778:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:3.0:1787826284.101788:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826284.101792:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826284.101794:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826284.101797:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346134144:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826284.101855:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826284.101970:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346134144 00000100:00100000:1.0:1787826284.101975:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346134144 00010000:00010000:1.0:1787826284.101997:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49ad3 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49acc expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826284.102224:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21869 00000100:00100000:1.0:1787826284.102234:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346134144:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826284.102247:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787826284.104146:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957327 is committed 00010000:00010000:1.0:1787826284.104328:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f49ad3 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49acc expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.104350:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961d81b00) 00010000:00010000:1.0:1787826284.104356:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.104367:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.104385:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.104514:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826284.104615:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346134144:12345-0@lo:103:ldlm_bl.0 Request processed in 2381us (2758us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.104626:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21869 00010000:00010000:2.0:1787826284.104711:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.104722:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826284.104916:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346134144:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826284.105055:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b752680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346134272:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826284.105102:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826284.105226:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826284.105293:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346134272 00000100:00100000:3.0:1787826284.105305:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346134272 00000100:00100000:2.0:1787826284.105605:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23375 00000100:00100000:2.0:1787826284.105617:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346134272:12345-0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826284.105864:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:24703:x1874663346134272:12345-0@lo:1000:mdt00_007.0 Request processed in 248us (760us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.105873:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23375 00000100:00100000:0.0:1787826284.105940:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346134272:0@lo:1000:mdt00_007.0 00010000:00010000:0.0:1787826284.106003:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49b19 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.106019:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49b19 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.106028:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49b19 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826284.106059:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49b19 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.106075:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.106082:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.106089:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.106096:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.106923:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826284.106932:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.106944:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107010:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826284.107014:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826284.107019:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107027:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49b0b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107046:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107054:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107062:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107068:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826284.107072:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107079:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107163:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961d81b00) 00010000:00010000:0.0:1787826284.107167:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826284.107171:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107177:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49b12 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107234:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49b04 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801051 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107251:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49b19 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801051 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.107260:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f794b57a500) 00010000:00010000:0.0:1787826284.107262:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826284.107265:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826284.107426:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843435880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223833:x1874663332539008:12345-192.168.203.56@tcp:101:bash.0 Request processed in 7522us (7673us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.107434:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68265 00000100:00100000:1.0:1787826284.110029:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826284.111949:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332539136 00000100:00100000:0.0:1787826284.111968:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332539136 00000100:00100000:0.0:1787826284.112050:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68266 00000100:00100000:0.0:1787826284.112063:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223833:x1874663332539136:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826284.112097:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826284.112483:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f49b20 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801058 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.112569:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49b27 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.112587:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49b27 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.112601:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49b27 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826284.112743:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e44:0x0]: returning size 4096 00000004:00200000:0.0:1787826284.112751:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826284.112806:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f49b20 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801058 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.112828:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49b27 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801058 expref: 9 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.112849:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:0.0:1787826284.112852:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826284.112857:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826284.112945:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223833:x1874663332539136:12345-192.168.203.56@tcp:101:bash.0 Request processed in 884us (2929us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.112953:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68266 00000100:00100000:1.0:1787826284.115908:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826284.115994:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332539264 00000100:00100000:0.0:1787826284.116009:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332539264 00000100:00100000:0.0:1787826284.116085:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68267 00000100:00100000:0.0:1787826284.116097:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223833:x1874663332539264:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826284.116161:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826284.116185:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f49b2e lrc: 2/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080105f expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.116262:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f49b35 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.116280:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f49b35 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.116294:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f49b35 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826284.116352:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x190:0x0]: returning size 4096 00000004:00200000:0.0:1787826284.116369:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826284.116413:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f49b2e lrc: 0/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080105f expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.116446:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f49b35 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080105f expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.116455:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:0.0:1787826284.116458:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826284.116463:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826284.116534:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223833:x1874663332539264:12345-192.168.203.56@tcp:101:bash.0 Request processed in 440us (640us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.116542:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68267 00000100:00100000:2.0:1787826284.119893:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826284.119989:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332539392 00000100:00100000:3.0:1787826284.120002:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332539392 00000100:00100000:3.0:1787826284.120078:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68268 00000100:00100000:3.0:1787826284.120090:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223833:x1874663332539392:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826284.120149:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826284.120176:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b3c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801066 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.120513:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.120532:0:35093:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49aef lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49ae8 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.120542:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.120589:0:35093:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49aef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49ae8 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826284.120607:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434a80 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000_UUID:35093:1874663346134400:0@lo:104: 00000100:00100000:3.0:1787826284.120670:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826284.120723:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7967909f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826284.120783:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346134400 00000100:00100000:1.0:1787826284.120789:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346134400 00000100:00100000:1.0:1787826284.120848:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7823 00000100:00100000:1.0:1787826284.120857:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35093:x1874663346134400:12345-0@lo:104: 00010000:00010000:1.0:1787826284.120872:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.120947:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826284.120955:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35093:x1874663346134400:12345-0@lo:104: Request processed in 100us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.120963:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7823 00010000:00010000:0.0:1787826284.121027:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826284.121037:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.121048:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826284.121055:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826284.121067:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.121102:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.121160:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826284.121186:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.121195:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f49ae8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49aef expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.121224:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826284.121226:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826284.121228:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826284.121248:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826284.121263:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346134528:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826284.121300:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.121344:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346134528 00000100:00100000:0.0:1787826284.121350:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346134528 00010000:00010000:0.0:1787826284.121385:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49aef lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49ae8 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826284.121412:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21870 00000100:00100000:0.0:1787826284.121420:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3652:x1874663346134528:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826284.121432:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826284.122343:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434a80 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000_UUID:35093:1874663346134400:0@lo:104: 00010000:00010000:3.0:1787826284.122866:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.123009:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f49aef lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49ae8 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.123025:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:0.0:1787826284.123031:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.123041:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.123049:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826284.123081:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826284.123517:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346134528:12345-0@lo:103:ldlm_bl.0 Request processed in 2095us (2212us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.123529:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21870 00000100:00100000:2.0:1787826284.124303:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346134528:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826284.124556:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.124569:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826284.124648:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18e:0x0]: returning size 4096 00000004:00200000:3.0:1787826284.124654:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826284.124718:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b3c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801066 expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.124741:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801066 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.124751:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:3.0:1787826284.124754:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826284.124757:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826284.124851:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223833:x1874663332539392:12345-192.168.203.56@tcp:101:bash.0 Request processed in 4762us (4972us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826284.124860:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68268 00000100:00100000:1.0:1787826284.133758:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826284.134324:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332539648 00000100:00100000:3.0:1787826284.134338:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332539648 00000100:00100000:3.0:1787826284.134420:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68269 00000100:00100000:3.0:1787826284.134434:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b752d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:223833:x1874663332539648:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826284.134462:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826284.134466:0:6529:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826284.134502:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f49b43 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801066 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.134711:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961d81400) 00010000:00010000:3.0:1787826284.134729:0:6529:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826284.134811:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49b4a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080106d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.135098:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.135262:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.135276:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.135293:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.135306:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.135318:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826284.135894:0:6529:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x51d:0x0], index 0: 0 00010000:00010000:2.0:1787826284.136073:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49b5f lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136085:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49b5f lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136093:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49b5f lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136276:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136285:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136292:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136300:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136306:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826284.136311:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136320:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136336:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826284.136339:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826284.136343:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136349:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49b51 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826284.136363:0:6529:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c4d49c28 reply state 00000000692bbd08 transno 8589957328 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136374:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136381:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f784abfe700) 00010000:00010000:2.0:1787826284.136383:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826284.136385:0:6529:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826284.136424:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49b4a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080106d expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136448:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49b5f lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080106d expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.136456:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:2.0:1787826284.136458:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826284.136460:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826284.136701:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b752d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223833:x1874663332539648:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2269us (2956us total) trans 8589957328 rc 0/0 00000100:00100000:2.0:1787826284.136710:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68269 00000100:00100000:0.0:1787826284.140098:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826284.140245:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332539776 00000100:00100000:2.0:1787826284.140258:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332539776 00000100:00100000:2.0:1787826284.140370:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30788 00000100:00100000:2.0:1787826284.140393:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:223833:x1874663332539776:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826284.140416:0:35097:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826284.140445:0:35097:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801074 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826284.140480:0:35097:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x51d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826284.140510:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=8096s v=(0 0) 00010000:00010000:2.0:1787826284.140516:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826284.140528:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 7 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.140695:0:35097:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 7 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.140713:0:35097:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826284.140844:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223833:x1874663332539776:12345-192.168.203.56@tcp:101:bash.0 Request processed in 445us (752us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.140853:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30788 00000100:00100000:0.0:1787826284.143431:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826284.143595:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332540032 00000100:00100000:3.0:1787826284.143613:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332540032 00000100:00100000:3.0:1787826284.143715:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9310 00000100:00100000:3.0:1787826284.143728:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:223833:x1874663332540032:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826284.144177:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223833:x1874663332540032:12345-192.168.203.56@tcp:35:bash.0 Request processed in 450us (763us total) trans 8589957329 rc 0/0 00000100:00100000:3.0:1787826284.144186:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9310 00000100:00100000:1.0:1787826284.146670:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826284.146832:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332540288 00000100:00100000:3.0:1787826284.146854:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332540288 00000100:00100000:3.0:1787826284.146920:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68270 00000100:00100000:3.0:1787826284.146932:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b752680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:223833:x1874663332540288:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826284.146958:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826284.146962:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826284.146999:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f49b35 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080105f expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.147024:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:3.0:1787826284.147030:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826284.147048:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f49b74 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080107b expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.147270:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 3/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.147285:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.147301:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.147316:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.147323:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.147330:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826284.147507:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434a80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346134656:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826284.147910:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826284.147956:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b1e8900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826284.148028:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346134656 00000100:00100000:2.0:1787826284.148037:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346134656 00000100:00100000:2.0:1787826284.148151:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23376 00000100:00100000:2.0:1787826284.148162:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35094:x1874663346134656:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826284.148273:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35094:x1874663346134656:12345-0@lo:1000:mdt00_011.0 Request processed in 112us (363us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.148281:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23376 00000100:00100000:1.0:1787826284.148369:0:35094:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957327 gen 2 00000004:00080000:1.0:1787826284.148376:0:35094:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957327 committed transno 8589957327 00000100:00100000:1.0:1787826284.148440:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434a80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346134656:0@lo:1000:mdt00_011.0 00000004:00080000:1.0:1787826284.148819:0:35094:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4c8:0x0], index 1: 0 00010000:00010000:1.0:1787826284.149037:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49b89 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149054:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49b89 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149065:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49b89 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149144:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149153:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149160:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149167:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149174:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826284.149179:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149187:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149199:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:1.0:1787826284.149202:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826284.149207:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 3/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149213:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f49b7b lrc: 0/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826284.149228:0:35094:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fa9105d8 reply state 000000005f69bfa0 transno 4294985346 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149241:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149248:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x4ec94fd6 (ffff9f79460a2800) 00010000:00010000:1.0:1787826284.149251:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826284.149253:0:35094:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826284.149297:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f49b74 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080107b expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149315:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49b89 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080107b expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.149329:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7967ac1400) 00010000:00010000:1.0:1787826284.149331:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826284.149334:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826284.149415:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b752680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223833:x1874663332540288:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2485us (2757us total) trans 4294985346 rc 0/0 00000100:00100000:1.0:1787826284.149423:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68270 00000100:00100000:0.0:1787826284.155895:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826284.156031:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332540416 00000100:00100000:1.0:1787826284.156044:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332540416 00000100:00100000:1.0:1787826284.156256:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30789 00000100:00100000:1.0:1787826284.156269:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:223833:x1874663332540416:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826284.156292:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826284.156320:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801082 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826284.156352:0:17826:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4c8:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787826284.156380:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c8:0x0].0x0 since=8096s v=(0 0) 00010000:00010000:1.0:1787826284.156385:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826284.156397:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 8 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.156431:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 8 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.156449:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826284.157174:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:223833:x1874663332540416:12345-192.168.203.56@tcp:101:bash.0 Request processed in 893us (1280us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.157185:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30789 00000100:00100000:0.0:1787826284.162719:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826284.162895:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332540672 00000100:00100000:3.0:1787826284.162909:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332540672 00000100:00100000:3.0:1787826284.162991:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9311 00000100:00100000:3.0:1787826284.163005:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:223833:x1874663332540672:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826284.163725:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:223833:x1874663332540672:12345-192.168.203.56@tcp:35:bash.0 Request processed in 718us (1018us total) trans 4294985347 rc 0/0 00000100:00100000:0.0:1787826284.163735:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9311 00000100:00100000:2.0:1787826284.176084:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826284.176371:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332540928 00000100:00100000:1.0:1787826284.176385:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332540928 00000100:00100000:2.0:1787826284.176765:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826284.176807:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826284.177834:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332540800 00000100:00100000:2.0:1787826284.177834:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332541056 00000100:00100000:1.0:1787826284.177857:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332540800 00000100:00100000:2.0:1787826284.177857:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332541056 00000100:00100000:3.0:1787826284.177865:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11018 00000100:00100000:3.0:1787826284.177884:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:6167:x1874663332540928:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826284.177905:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826284.177918:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11019 00000100:00100000:1.0:1787826284.177929:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:34880:x1874663332540800:12345-192.168.203.56@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826284.177933:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11020 00010000:00010000:1.0:1787826284.177942:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826284.177943:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4997c lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1db5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800ff6 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826284.177950:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:34881:x1874663332541056:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826284.177964:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49afd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080104a expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.177969:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826284.177978:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db5:0x0].0x0 (ffff9f794b57af00) 00010000:00010000:1.0:1787826284.177985:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826284.177993:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49b89 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080107b expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826284.177997:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826284.178003:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49b5f lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080106d expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.178018:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:1.0:1787826284.178035:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f49b19 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801051 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826284.178046:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f794b57a500) 00010000:00010000:1.0:1787826284.178053:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826284.178075:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:6167:x1874663332540928:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 197us (2004us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826284.178084:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:34880:x1874663332540800:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 156us (1320us total) trans 0 rc 0/0 00010000:00010000:2.0:1787826284.178085:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7967ac1400) 00000100:00100000:3.0:1787826284.178089:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11018 00000100:00100000:1.0:1787826284.178101:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11019 00010000:00010000:2.0:1787826284.178207:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49b27 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801058 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.178222:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:2.0:1787826284.178387:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f498db lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e42:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800fda expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826284.178401:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e42:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:2.0:1787826284.178408:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826284.178446:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:34881:x1874663332541056:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 502us (1638us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.178454:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11020 00000100:00100000:0.0:1787826284.524473:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346134784:0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.524583:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.524596:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346135168:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.524642:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.524652:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346135296:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.524674:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346135680:0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.524707:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.524738:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346135808:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.524742:0:30904:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346134784 00000100:00100000:2.0:1787826284.524752:0:30904:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346134784 00000100:00100000:0.0:1787826284.524765:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.524788:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1787826284.524796:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346134912:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.524819:0:30904:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2211 00000100:00100000:0.0:1787826284.524825:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826284.524844:0:30904:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847800450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346134784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.524857:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346135424:0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.524885:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.524904:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346135936:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.524918:0:30904:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847800450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346134784:12345-0@lo:400:kworker.0 Request processed in 86us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.524928:0:30904:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2211 00000100:00100000:0.0:1787826284.524933:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.524971:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346134784:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.524991:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346135168 00000100:00100000:2.0:1787826284.524997:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346135168 00000100:00100000:0.0:1787826284.525007:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.525034:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1787826284.525041:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346135040:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.525046:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346134912 00000100:00100000:2.0:1787826284.525051:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346134912 00000100:00100000:0.0:1787826284.525070:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.525093:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346135552:0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.525165:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826284.525233:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346135296 00000100:00100000:3.0:1787826284.525247:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346135296 00000100:00100000:3.0:1787826284.525363:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346135808 00000100:00100000:3.0:1787826284.525367:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346135808 00000100:00100000:3.0:1787826284.525395:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346135936 00000100:00100000:3.0:1787826284.525398:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346135936 00000100:00100000:3.0:1787826284.525418:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346135680 00000100:00100000:3.0:1787826284.525420:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346135680 00000100:00100000:3.0:1787826284.525444:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8773 00000100:00100000:3.0:1787826284.525454:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346135296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826284.525503:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346135296:12345-0@lo:400:kworker.0 Request processed in 51us (797us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826284.525511:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8773 00000100:00100000:3.0:1787826284.525526:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8774 00000100:00100000:3.0:1787826284.525533:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346135808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826284.525561:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346135808:12345-0@lo:400:kworker.0 Request processed in 29us (797us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826284.525568:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8774 00000100:00100000:3.0:1787826284.525581:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8775 00000100:00100000:3.0:1787826284.525587:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346135936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826284.525617:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346135936:12345-0@lo:400:kworker.0 Request processed in 30us (685us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826284.525623:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8775 00000100:00100000:3.0:1787826284.525634:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8776 00000100:00100000:3.0:1787826284.525640:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346135680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826284.525668:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346135680:12345-0@lo:400:kworker.0 Request processed in 29us (662us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826284.525674:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8776 00000100:00100000:2.0:1787826284.527038:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346135424 00000100:00100000:2.0:1787826284.527054:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346135424 00000100:00100000:2.0:1787826284.527141:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346135040 00000100:00100000:2.0:1787826284.527145:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346135040 00000100:00100000:2.0:1787826284.527176:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14243 00000100:00100000:2.0:1787826284.527188:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346135168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.529338:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346135168:12345-0@lo:400:kworker.0 Request processed in 2139us (4682us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.529359:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14243 00000100:00100000:2.0:1787826284.529416:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14244 00000100:00100000:2.0:1787826284.529431:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346134912:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.529628:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346136064:0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.529678:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826284.529730:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346135168:0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.529748:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346135680:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.529784:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346134912:12345-0@lo:400:kworker.0 Request processed in 357us (4959us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.529793:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14244 00000100:00100000:0.0:1787826284.529808:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346135552 00000100:00100000:2.0:1787826284.529812:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346136064 00000100:00100000:0.0:1787826284.529817:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346135552 00000100:00100000:2.0:1787826284.529818:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346136064 00000100:00100000:2.0:1787826284.529872:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14245 00000100:00100000:2.0:1787826284.529882:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346135424:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.529888:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14246 00000100:00100000:0.0:1787826284.529900:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346135040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.529923:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346135424:12345-0@lo:400:kworker.0 Request processed in 43us (5037us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.529931:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14245 00000100:00100000:2.0:1787826284.529951:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14247 00000100:00100000:0.0:1787826284.529956:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346135040:12345-0@lo:400:kworker.0 Request processed in 57us (4885us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.529964:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346136064:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.529969:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14246 00000100:00100000:0.0:1787826284.529992:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14248 00000100:00100000:0.0:1787826284.530000:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346135552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.530009:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346136064:12345-0@lo:400:kworker.0 Request processed in 47us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826284.530016:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14247 00000100:00100000:0.0:1787826284.530041:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346135552:12345-0@lo:400:kworker.0 Request processed in 42us (4874us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826284.530048:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14248 00000100:00080000:2.0:1787826284.530053:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826284.530061:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826284.530067:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826284.530070:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826284.530151:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346135040:0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.530165:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346135552:0@lo:400:kworker.0 00000100:00100000:0.0:1787826284.530196:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346136064:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.531045:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346134912:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.531067:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346135424:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.531082:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346135936:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.531095:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346135296:0@lo:400:kworker.0 00000100:00100000:2.0:1787826284.531145:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973836 gen 2 00000004:00080000:2.0:1787826284.531152:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784b759c00, transno 4294973836 00000100:00100000:2.0:1787826284.531165:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346135808:0@lo:400:kworker.0 00010000:00010000:1.0:1787826284.729252:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826285.037761:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346136192:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826285.037864:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826285.037941:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346136192 00000100:00100000:3.0:1787826285.037950:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346136192 00000100:00100000:3.0:1787826285.038012:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6203 00000100:00100000:3.0:1787826285.038023:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346136192:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826285.038060:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826285.038126:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346136192:12345-0@lo:13:osp-pre-1-1.0 Request processed in 104us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826285.038135:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6203 00000100:00100000:3.0:1787826285.038196:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346136192:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1787826285.397069:0:165076:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:3.0:1787826285.397210:0:165076:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000100:00100000:3.0:1787826285.548259:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826285.548283:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346136320:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826285.548355:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826285.548407:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346136320 00000100:00100000:3.0:1787826285.548415:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346136320 00000100:00100000:3.0:1787826285.548463:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23377 00000100:00100000:3.0:1787826285.548473:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346136320:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787826285.548506:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443426304 avail 1297186816 00000100:00100000:3.0:1787826285.548551:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346136320:12345-0@lo:41:osp-pre-1-0.0 Request processed in 79us (196us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826285.548558:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23377 00000100:00100000:3.0:1787826285.548602:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346136320:0@lo:41:osp-pre-1-0.0 00000004:00080000:1.0:1787826285.549253:0:8751:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8097761146008ns, next at 8098761143165ns, rc = 0 00000100:00100000:1.0:1787826285.549295:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346136576:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826285.549336:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826285.549365:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826285.549373:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346136704:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826285.549400:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826285.549481:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346136576 00000100:00100000:3.0:1787826285.549485:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346136576 00000100:00100000:3.0:1787826285.549522:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346136704 00000100:00100000:3.0:1787826285.549526:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346136704 00000100:00100000:3.0:1787826285.549541:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6204 00000100:00100000:3.0:1787826285.549547:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346136576:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826285.549568:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662454784 00000100:00100000:3.0:1787826285.549611:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346136576:12345-0@lo:13:osp-pre-0-1.0 Request processed in 63us (275us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826285.549617:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6204 00000100:00100000:3.0:1787826285.549632:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6205 00000100:00100000:3.0:1787826285.549639:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346136704:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787826285.549649:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662454784 00000100:00100000:3.0:1787826285.549676:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346136704:12345-0@lo:13:osp-pre-0-0.0 Request processed in 38us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826285.549750:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6205 00000100:00100000:3.0:1787826285.549787:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346136704:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826285.549798:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346136576:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826285.549821:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346136448:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826285.549857:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826285.549878:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826285.549884:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346136832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826285.549910:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826285.549937:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346136448 00000100:00100000:3.0:1787826285.549940:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346136448 00000100:00100000:3.0:1787826285.549969:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23378 00000100:00100000:3.0:1787826285.549976:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346136448:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787826285.549996:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442820096 avail 1296580608 00000100:00100000:3.0:1787826285.550030:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346136448:12345-0@lo:41:osp-pre-0-1.0 Request processed in 55us (172us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826285.550036:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23378 00000100:00100000:1.0:1787826285.550148:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346136832 00000100:00100000:1.0:1787826285.550152:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346136832 00000100:00100000:1.0:1787826285.550175:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6206 00000100:00100000:1.0:1787826285.550181:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346136832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826285.550192:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826285.550225:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346136832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 43us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826285.550231:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6206 00000100:00100000:3.0:1787826285.550609:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346136832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826285.550623:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346136448:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1787826285.740483:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826286.764953:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1787826287.598755:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 362 committed 00010000:00010000:2.0:1787826287.598810:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49a24 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826287.598838:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49a24 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.598905:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a24 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.598914:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826287.598941:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f49a1d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49a24 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826287.598965:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985347 is committed 00000020:00080000:2.0:1787826287.598986:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784b4df3c0 refcount 2 committed 0 result 0 batchid 3397 00000020:00080000:2.0:1787826287.598990:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784b4dfa80 00000020:00080000:2.0:1787826287.598994:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1500 00000020:00080000:2.0:1787826287.598997:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff200 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78525efe00 00000020:00080000:2.0:1787826287.599000:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1181 00000001:00080000:2.0:1787826287.599013:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985344 is committed 00000020:00080000:2.0:1787826287.599017:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784548a3c0 refcount 2 committed 0 result 0 batchid 3396 00000020:00080000:2.0:1787826287.599019:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78434d1900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784548a9c0 00000020:00080000:2.0:1787826287.599023:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1499 00000020:00080000:2.0:1787826287.599026:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78434d1400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78445d5a00 00000020:00080000:2.0:1787826287.599028:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1180 00000004:00080000:2.0:1787826287.599037:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 361 committed 00000020:00080000:2.0:1787826287.599041:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784b4df0c0 refcount 2 committed 0 result 0 batchid 3395 00000020:00080000:2.0:1787826287.599044:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78448e0e00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f796cce2f00 00000020:00080000:2.0:1787826287.599047:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1498 00000020:00080000:2.0:1787826287.599050:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78448e0300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78525ed400 00000020:00080000:2.0:1787826287.599053:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1179 00000020:00080000:2.0:1787826287.599060:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7852405600 refcount 2 committed 0 result 0 batchid 3394 00000020:00080000:2.0:1787826287.599062:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78448e0800 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78524059c0 00000020:00080000:2.0:1787826287.599065:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1497 00000020:00080000:2.0:1787826287.599067:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78448e0600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78525eca00 00000020:00080000:2.0:1787826287.599070:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1178 00000001:00080000:2.0:1787826287.599075:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985338 is committed 00000020:00080000:2.0:1787826287.599099:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3393 batchid updated 00010000:00010000:2.0:1787826287.599219:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599229:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599237:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599243:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826287.599248:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599255:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599272:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:2.0:1787826287.599276:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826287.599281:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599287:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f49ada lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599306:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599312:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599319:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599324:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826287.599327:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599333:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599340:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:2.0:1787826287.599342:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826287.599345:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599351:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f49ae1 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599370:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599377:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599384:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599388:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826287.599390:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599396:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599404:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x4ec94fd6 (ffff9f79460a2800) 00010000:00010000:2.0:1787826287.599406:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826287.599409:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.599415:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f49b82 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826287.599457:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346136960:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787826287.599551:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:2.0:1787826287.599644:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3397 committed batchid 3393 00000004:00080000:2.0:1787826287.599772:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78453cef00 with version 6792 00000020:00080000:2.0:1787826287.599783:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3390 cancel update log [0x200000401:0x3:0x0].1492: rc = 0 00000020:00080000:2.0:1787826287.599849:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3390 cancel update log [0x240000400:0x3:0x0].1174: rc = 0 00000004:00080000:2.0:1787826287.599879:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78453cec00 with version 6793 00000020:00080000:2.0:1787826287.599885:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3391 cancel update log [0x200000401:0x3:0x0].1494: rc = 0 00000020:00080000:2.0:1787826287.599908:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3391 cancel update log [0x240000400:0x3:0x0].1175: rc = 0 00000004:00080000:2.0:1787826287.599931:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7849c3b480 with version 6794 00000020:00080000:2.0:1787826287.599935:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3392 cancel update log [0x200000401:0x3:0x0].1495: rc = 0 00000020:00080000:2.0:1787826287.599950:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3392 cancel update log [0x240000400:0x3:0x0].1176: rc = 0 00000004:00080000:2.0:1787826287.599969:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7850703e40 with version 6795 00000020:00080000:2.0:1787826287.599972:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3393 cancel update log [0x200000401:0x3:0x0].1496: rc = 0 00000020:00080000:2.0:1787826287.599992:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3393 cancel update log [0x240000400:0x3:0x0].1177: rc = 0 00000020:00080000:2.0:1787826287.599998:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3393 00000100:00100000:2.0:1787826287.600019:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346136960 00000100:00100000:2.0:1787826287.600027:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346136960 00000100:00100000:2.0:1787826287.600090:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11021 00000100:00100000:2.0:1787826287.600101:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c83050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346136960:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826287.600141:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826287.600168:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f49a24 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18d:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49a1d expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.600183:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18d:0x0].0x5a734ce9 (ffff9f794b57a700) 00010000:00010000:2.0:1787826287.600193:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:1.0:1787826287.602034:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6792 rpc_version 6792 00000020:00100000:1.0:1787826287.602133:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826287.602223:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c83050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346136960:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 146us (693us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826287.602238:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11021 00000100:00100000:2.0:1787826287.602387:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346136960:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826287.603430:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc32680 x1874663346137344/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826287.603465:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346137344:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826287.604059:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826287.604216:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f797cd239c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826287.604240:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826287.604257:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346137088:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826287.604291:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826287.604642:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346137088 00000100:00100000:0.0:1787826287.604658:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346137088 00000100:00100000:0.0:1787826287.604780:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30790 00000100:00100000:0.0:1787826287.604800:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346137088:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1787826287.604842:0:8759:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1226 count 1 00010000:00010000:0.0:1787826287.604870:0:8759:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.604897:0:8759:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.604912:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4ca:0x0].0x0 since=9s v=(0 0) 00010000:00010000:0.0:1787826287.604918:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826287.604923:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.604965:0:8759:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.604989:0:8759:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 6 pid: 35096 timeout: 8199 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826287.605786:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976147 is committed 00000001:00080000:1.0:1787826287.605798:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976146 is committed 00000001:00100000:1.0:1787826287.605809:0:9524:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000001:00080000:1.0:1787826287.605828:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976145 is committed 00000100:00100000:0.0:1787826287.605869:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346137472:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826287.605984:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057780 going to sleep for 11 seconds 00000100:00080000:1.0:1787826287.608064:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826287.612303:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346137344 00000100:00100000:3.0:1787826287.612326:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346137344 00000100:00100000:3.0:1787826287.612664:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23379 00000100:00100000:3.0:1787826287.612676:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346137344:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826287.612996:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346137344:12345-0@lo:1000:osp_up0-1.0 Request processed in 322us (8939us total) trans 8589957330 rc 0/0 00000100:00100000:3.0:1787826287.613005:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23379 00000100:00100000:3.0:1787826287.613665:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346137344:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826287.613686:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6793 rpc_version 6793 00000020:00100000:3.0:1787826287.613719:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826287.613727:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7843434700 x1874663346137728/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826287.613752:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346137728:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826287.613825:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826287.613858:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826287.614062:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346137728 00000100:00100000:3.0:1787826287.614070:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346137728 00000100:00100000:3.0:1787826287.614166:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23380 00000100:00100000:3.0:1787826287.614178:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346137728:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826287.614352:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346137728:12345-0@lo:1000:osp_up0-1.0 Request processed in 176us (541us total) trans 8589957331 rc 0/0 00000100:00100000:3.0:1787826287.614360:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23380 00000100:00100000:3.0:1787826287.614410:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346137728:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826287.614420:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6794 rpc_version 6794 00000020:00100000:3.0:1787826287.614437:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826287.614442:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78435b3800 x1874663346137984/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826287.614458:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346137984:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826287.614505:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826287.614525:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826287.616767:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826287.622049:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332541184 00000100:00100000:0.0:1787826287.622069:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332541184 00010000:00010000:0.0:1787826287.622721:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 7 pid: 35096 timeout: 8199 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826287.622773:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21871 00000100:00100000:0.0:1787826287.622785:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332541184:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826287.622807:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826287.622826:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 7 pid: 35096 timeout: 8199 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.622848:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 7 pid: 35096 timeout: 8199 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826287.623157:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346137984 00000100:00100000:1.0:1787826287.623167:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346137984 00000100:00100000:1.0:1787826287.623225:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23381 00000100:00100000:1.0:1787826287.623234:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346137984:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826287.623717:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346137984:12345-0@lo:1000:osp_up0-1.0 Request processed in 483us (9209us total) trans 8589957332 rc 0/0 00000100:00100000:1.0:1787826287.623726:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23381 00010000:00010000:0.0:1787826287.623777:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826287.623785:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826287.623792:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.623806:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.623843:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826287.623904:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332541184:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1120us (7148us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826287.623919:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21871 00000100:00080000:0.0:1787826287.623962:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787826287.624140:0:8759:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d5fee53/0xe21309aa81f49833 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760800f9b expref: 6 pid: 35096 timeout: 8199 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826287.624159:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346137472:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826287.624185:0:8759:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826287.624406:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346137984:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826287.624421:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6795 rpc_version 6795 00000020:00100000:3.0:1787826287.624450:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826287.624458:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b759880 x1874663346138240/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826287.624480:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346138240:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826287.624533:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826287.624563:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826287.624584:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346138240 00000100:00100000:3.0:1787826287.624590:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346138240 00000100:00100000:3.0:1787826287.624632:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23382 00000100:00100000:3.0:1787826287.624642:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346138240:12345-0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826287.624719:0:8759:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.624734:0:8759:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.624746:0:8759:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826287.624767:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346138240:12345-0@lo:1000:osp_up0-1.0 Request processed in 125us (232us total) trans 8589957333 rc 0/0 00010000:00010000:1.0:1787826287.624771:0:8759:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826287.624782:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23382 00010000:00010000:1.0:1787826287.624806:0:8759:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826287.624839:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346138240:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826287.625012:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346137088:12345-0@lo:6:osp-syn-0-1.0 Request processed in 20222us (20719us total) trans 4294973838 rc 0/0 00000100:00100000:1.0:1787826287.625021:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30790 00010000:00010000:0.0:1787826287.626341:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826287.626360:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.626375:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826287.626380:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.626393:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.626420:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.626428:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4243488/0xe21309aa81f49b9e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.626454:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826287.626456:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826287.626458:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1787826287.626846:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784c6bb480/2, rc 0, transno 4294973838 00000100:00100000:3.0:1787826287.626858:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346137088:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826287.761779:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826287.762316:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332541312 00000100:00100000:3.0:1787826287.762344:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332541312 00000100:00100000:3.0:1787826287.762451:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68271 00000100:00100000:3.0:1787826287.762466:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224014:x1874663332541312:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826287.762501:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826287.762531:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f49ba5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801089 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826287.762630:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826287.762651:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826287.762668:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826287.762740:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826287.762745:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826287.762774:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826287.762807:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f49ba5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826287.762837:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826287.762848:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826287.762851:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826287.762857:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826287.762939:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224014:x1874663332541312:12345-192.168.203.56@tcp:101:cat.0 Request processed in 475us (1175us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826287.762947:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68271 00000100:00080000:2.0:1787826287.763364:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826287.769437:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826287.772997:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332541440 00000100:00100000:2.0:1787826287.773015:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332541440 00000100:00100000:2.0:1787826287.773133:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68272 00000100:00100000:2.0:1787826287.773147:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224014:x1874663332541440:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826287.773177:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826287.773205:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49bb3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801090 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773296:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773315:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773340:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773361:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773368:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773374:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773468:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49bc8 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773478:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49bc8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773484:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49bc8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826287.773515:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49bc8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773533:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773541:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773547:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773553:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773559:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826287.773565:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773573:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773589:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826287.773593:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826287.773599:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773606:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49bba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773626:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773634:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773641:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773646:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826287.773649:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773656:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773662:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9900) 00010000:00010000:2.0:1787826287.773665:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826287.773667:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773673:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49bc1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773711:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49bb3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801090 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773727:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49bc8 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801090 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826287.773735:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:2.0:1787826287.773738:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826287.773741:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826287.773843:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224014:x1874663332541440:12345-192.168.203.56@tcp:101:cat.0 Request processed in 697us (4419us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826287.773851:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68272 00000100:00100000:3.0:1787826287.778126:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826287.779455:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332541568 00000100:00100000:1.0:1787826287.779473:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332541568 00000100:00100000:1.0:1787826287.779693:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68273 00000100:00100000:1.0:1787826287.779709:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224014:x1874663332541568:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826287.779746:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826287.779779:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49bcf lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801097 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.779875:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f49bd6 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.779893:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f49bd6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.779906:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f49bd6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826287.780037:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e44:0x0]: returning size 4096 00000004:00200000:1.0:1787826287.780043:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826287.780229:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49bcf lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801097 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.780256:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f49bd6 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801097 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.780267:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7961614600) 00010000:00010000:1.0:1787826287.780271:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826287.780275:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826287.780434:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224014:x1874663332541568:12345-192.168.203.56@tcp:101:cat.0 Request processed in 728us (2340us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826287.780443:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68273 00000100:00080000:3.0:1787826287.782821:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:1.0:1787826287.788347:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826287.790424:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826287.792654:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332541696 00000100:00100000:1.0:1787826287.792674:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332541696 00000100:00100000:1.0:1787826287.792769:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68274 00000100:00100000:1.0:1787826287.792785:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224014:x1874663332541696:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826287.792825:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826287.792863:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49bdd lrc: 2/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080109e expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.792951:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.792969:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.792986:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826287.793052:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x190:0x0]: returning size 4096 00000004:00200000:1.0:1787826287.793059:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826287.793146:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49bdd lrc: 0/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.793168:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.793178:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f7961614400) 00010000:00010000:1.0:1787826287.793182:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826287.793187:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826287.793426:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224014:x1874663332541696:12345-192.168.203.56@tcp:101:cat.0 Request processed in 644us (3013us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826287.793435:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68274 00000100:00100000:3.0:1787826287.796211:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826287.796309:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332541824 00000100:00100000:0.0:1787826287.796324:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332541824 00000100:00100000:0.0:1787826287.796413:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68275 00000100:00100000:0.0:1787826287.796427:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224014:x1874663332541824:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826287.796459:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826287.796490:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f49beb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010a5 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.796573:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.796590:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.796605:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826287.796673:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18e:0x0]: returning size 4096 00000004:00200000:0.0:1787826287.796679:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826287.796721:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f49beb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.796739:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826287.796748:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:0.0:1787826287.796751:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826287.796755:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826287.797249:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224014:x1874663332541824:12345-192.168.203.56@tcp:101:cat.0 Request processed in 821us (1051us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826287.797258:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68275 00000100:00100000:2.0:1787826287.800452:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826287.800547:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332542080 00000100:00100000:1.0:1787826287.800559:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332542080 00000100:00100000:1.0:1787826287.800630:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68276 00000100:00100000:1.0:1787826287.800643:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224014:x1874663332542080:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826287.800671:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826287.800697:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49bf9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010ac expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.800796:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.800822:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.800840:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.800853:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.800861:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826287.800869:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:1.0:1787826287.800963:0:24703:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000100:00100000:0.0:1787826288.277754:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826288.277882:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332542208 00000100:00100000:1.0:1787826288.277897:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332542208 00000100:00080000:3.0:1787826288.277986:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826288.278003:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68277 00000100:00100000:1.0:1787826288.278019:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224016:x1874663332542208:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826288.278056:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826288.278082:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49c0e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010b3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.278222:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.278240:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.278254:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826288.278359:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826288.278366:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826288.278404:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.278439:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49c0e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.278464:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.278474:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826288.278478:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826288.278594:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:0.0:1787826288.278733:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224016:x1874663332542208:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 718us (994us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826288.278742:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68277 00000100:00100000:2.0:1787826288.284966:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826288.290987:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332542336 00000100:00100000:3.0:1787826288.291007:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332542336 00000100:00100000:3.0:1787826288.291130:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68278 00000100:00100000:3.0:1787826288.291145:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224016:x1874663332542336:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826288.291173:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826288.291200:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f49c1c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010ba expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291299:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291316:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291330:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291349:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291357:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291364:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291464:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291473:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291481:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826288.291521:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291538:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291547:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291554:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291560:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291567:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826288.291572:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291580:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291595:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826288.291598:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826288.291604:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291611:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c23 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291629:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291635:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291642:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291648:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826288.291650:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291656:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291663:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79460a2b00) 00010000:00010000:3.0:1787826288.291666:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826288.291668:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291675:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c2a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291712:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f49c1c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291727:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.291734:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:3.0:1787826288.291737:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826288.291740:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826288.291844:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224016:x1874663332542336:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 702us (6895us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826288.291853:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68278 00000100:00100000:1.0:1787826288.300769:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826288.304530:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332542464 00000100:00100000:2.0:1787826288.304554:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332542464 00000100:00080000:3.0:1787826288.305413:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826288.305585:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68279 00000100:00100000:2.0:1787826288.305604:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:224016:x1874663332542464:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826288.305654:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826288.305682:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f49c38 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010c1 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.305765:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49c3f lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.305782:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49c3f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.305796:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49c3f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826288.305949:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e44:0x0]: returning size 4096 00000004:00200000:2.0:1787826288.305956:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826288.306008:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f49c38 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010c1 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.306032:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49c3f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010c1 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.306043:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7961614600) 00010000:00010000:2.0:1787826288.306047:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826288.306051:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826288.306188:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224016:x1874663332542464:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 590us (5433us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.306197:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68279 00000100:00100000:0.0:1787826288.316223:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826288.318334:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332542592 00000100:00100000:1.0:1787826288.318356:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332542592 00000100:00100000:1.0:1787826288.318454:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68280 00000100:00100000:1.0:1787826288.318468:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224016:x1874663332542592:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826288.318507:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826288.318536:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c46 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010c8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318628:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 3/1,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318646:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318660:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318680:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318688:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318695:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318818:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49c5b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318832:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49c5b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318840:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49c5b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826288.318895:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e45:0x0]: returning size 6 00000004:00200000:1.0:1787826288.318900:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826288.318925:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49c5b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318940:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49c5b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318951:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318958:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318965:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318972:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318979:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826288.318984:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.318992:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319008:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f7961614400) 00010000:00010000:1.0:1787826288.319011:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826288.319016:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 3/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319023:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f49c4d lrc: 0/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319039:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319045:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319050:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319055:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826288.319058:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319063:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319070:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x4ec94fd6 (ffff9f7961614b00) 00010000:00010000:1.0:1787826288.319072:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826288.319075:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319080:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49c54 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319189:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c46 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010c8 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319205:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49c5b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010c8 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.319212:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7961614200) 00010000:00010000:1.0:1787826288.319215:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826288.319217:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:1.0:1787826288.319312:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224016:x1874663332542592:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 845us (3101us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.319320:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68280 00000100:00100000:1.0:1787826288.334963:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826288.335076:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332542720 00000100:00100000:3.0:1787826288.335091:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332542720 00000100:00100000:3.0:1787826288.335197:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68281 00000100:00100000:3.0:1787826288.335209:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224016:x1874663332542720:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826288.335245:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826288.335271:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f49c62 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010cf expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335527:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335546:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335560:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335571:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335579:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335585:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335708:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49c77 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335718:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49c77 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335730:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49c77 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826288.335784:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db6:0x0]: returning size 6 00000004:00200000:3.0:1787826288.335790:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826288.335828:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49c77 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335844:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49c77 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335856:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335862:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335868:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335875:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335881:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826288.335886:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335894:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335909:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:3.0:1787826288.335913:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826288.335920:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335927:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49c69 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335946:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335951:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335957:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335963:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826288.335965:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335971:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335979:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f784abfe700) 00010000:00010000:3.0:1787826288.335981:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826288.335984:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.335990:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f49c70 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.336024:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f49c62 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010cf expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.336042:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49c77 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010cf expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.336050:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826288.336052:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826288.336054:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:3.0:1787826288.336190:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224016:x1874663332542720:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 982us (1245us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826288.336200:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68281 00000100:00100000:3.0:1787826288.341646:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826288.341701:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826288.341750:0:125380:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332543104 00000100:00100000:0.0:1787826288.341763:0:125380:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332543104 00000100:00100000:2.0:1787826288.341789:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332542848 00000100:00100000:2.0:1787826288.341794:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332542848 00000100:00100000:0.0:1787826288.341859:0:125380:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1225 00000100:00100000:2.0:1787826288.341859:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11022 00000100:00100000:0.0:1787826288.341871:0:125380:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224016:x1874663332543104:12345-192.168.203.56@tcp:36:mrename.0 00000100:00100000:2.0:1787826288.341871:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332542848:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:2.0:1787826288.341892:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826288.341926:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49c5b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010c8 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.341933:0:125380:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826288.341949:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f49c77 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010cf expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.341951:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7961614200) 00010000:00010000:0.0:1787826288.341961:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:2.0:1787826288.341965:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826288.341969:0:125380:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826288.342034:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332542848:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 165us (332us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.342041:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11022 00000100:00100000:0.0:1787826288.342081:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138368:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.342178:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826288.342215:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3b0f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826288.342238:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346138368 00000100:00100000:0.0:1787826288.342242:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346138368 00000100:00100000:2.0:1787826288.342292:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23383 00000100:00100000:2.0:1787826288.342303:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138368:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.342384:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138368:12345-0@lo:1000:mdt_io00_003.0 Request processed in 83us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.342392:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23383 00000100:00100000:1.0:1787826288.342452:0:125380:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985338 gen 2 00000004:00080000:1.0:1787826288.342460:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985337 committed transno 4294985338 00000020:00080000:1.0:1787826288.342468:0:125380:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f78524050c0 refcount 2 committed 0 result 0 batchid 4294979077 00000020:00080000:1.0:1787826288.342472:0:125380:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78448e0900 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852405240 00000020:00080000:1.0:1787826288.342476:0:125380:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].933 00000020:00080000:1.0:1787826288.342479:0:125380:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7946083600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784c4cd000 00000020:00080000:1.0:1787826288.342482:0:125380:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].930 00000004:00080000:1.0:1787826288.342516:0:125380:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985338 committed transno 4294985338 00000100:00100000:1.0:1787826288.342537:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138368:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.342607:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc30000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138496:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.342651:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826288.342673:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453d9900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826288.342757:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346138496 00000100:00100000:0.0:1787826288.342762:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346138496 00000020:00080000:2.0:1787826288.344251:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979077 committed batchid 4294979076 00000020:00080000:2.0:1787826288.345406:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979076 00000100:00100000:0.0:1787826288.345774:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23384 00000100:00100000:0.0:1787826288.345799:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138496:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.346025:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138496:12345-0@lo:1000:mdt_io00_003.0 Request processed in 236us (3383us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826288.346039:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23384 00000100:00100000:2.0:1787826288.346172:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc30000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138496:0@lo:1000:mdt_io00_003.0 00000004:00080000:2.0:1787826288.346228:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x190:0x0] 00000100:00100000:2.0:1787826288.346311:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138624:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.350468:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826288.359560:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f797cd239c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826288.359651:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346138624 00000100:00100000:1.0:1787826288.359664:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346138624 00000100:00100000:1.0:1787826288.359743:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23385 00000100:00100000:1.0:1787826288.359756:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138624:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.359934:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138624:12345-0@lo:1000:mdt_io00_003.0 Request processed in 180us (9479us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.359943:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23385 00000100:00100000:1.0:1787826288.364651:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138624:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826288.364726:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e44:0x0] 00000100:00100000:1.0:1787826288.364823:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138752:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.364912:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826288.364955:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826288.364983:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346138752 00000100:00100000:1.0:1787826288.364991:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346138752 00000100:00100000:1.0:1787826288.365049:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23386 00000100:00100000:1.0:1787826288.365061:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138752:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.365194:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138752:12345-0@lo:1000:mdt_io00_003.0 Request processed in 134us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.365203:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23386 00000100:00100000:1.0:1787826288.365239:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138752:0@lo:1000:mdt_io00_003.0 00000004:00080000:0.0:1787826288.368410:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x190:0x0] 00000100:00100000:0.0:1787826288.368525:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138880:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.368621:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826288.368673:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3b0480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826288.368744:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346138880 00000100:00100000:1.0:1787826288.368754:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346138880 00000100:00100000:1.0:1787826288.368831:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23387 00000100:00100000:1.0:1787826288.368843:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138880:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.368974:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346138880:12345-0@lo:1000:mdt_io00_003.0 Request processed in 131us (352us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.368983:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23387 00000100:00100000:3.0:1787826288.369091:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346138880:0@lo:1000:mdt_io00_003.0 00000004:00020000:3.0:1787826288.369195:0:125380:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:3.0:1787826288.369247:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 3/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.369257:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 3/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.369268:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 3/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826288.369287:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346139008:0@lo:101:mdt_io00_003.0 00000100:00100000:3.0:1787826288.369348:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.369402:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346139008 00000100:00100000:3.0:1787826288.369409:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346139008 00000100:00100000:3.0:1787826288.369464:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23388 00000100:00100000:3.0:1787826288.369476:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346139008:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:3.0:1787826288.369492:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826288.369507:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49c7e expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.369532:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 6 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.369542:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 4/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49c7e expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.369563:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 6 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.369575:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 6 pid: 11629 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826288.369586:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6533:1874663346139136:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826288.369649:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826288.369672:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753be40 going to sleep for 11 seconds 00000004:00020000:1.0:1787826288.370722:0:24703:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4430 00010000:00010000:1.0:1787826288.370836:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.370856:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.370870:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826288.370884:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db6:0x0]: returning size 6 00010000:00010000:1.0:1787826288.370998:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371008:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371015:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371023:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371030:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826288.371034:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371043:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371059:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:1.0:1787826288.371063:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826288.371069:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371076:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49c00 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371095:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371144:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371151:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371157:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826288.371160:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371167:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371176:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f784abfe700) 00010000:00010000:1.0:1787826288.371178:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826288.371182:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371188:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49c07 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371245:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49bf9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 12 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371262:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.371270:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f7961614b00) 00010000:00010000:1.0:1787826288.371273:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826288.374577:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:0.0:1787826288.374823:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224014:x1874663332542080:12345-192.168.203.56@tcp:101:cat.0 Request processed in 574164us (574370us total) trans 8589957334 rc 0/0 00000100:00100000:0.0:1787826288.374834:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68276 00000100:00100000:3.0:1787826288.380306:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6533:1874663346139136:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826288.380366:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49c7e expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.380382:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826288.380385:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826288.380390:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49c7e expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.380402:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826288.380491:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346139008:12345-0@lo:101:mdt_io00_003.0 Request processed in 11016us (11141us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826288.380500:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23388 00000100:00100000:0.0:1787826288.380671:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346139008:0@lo:101:mdt_io00_003.0 00010000:00010000:0.0:1787826288.380692:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 4/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.380704:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 4/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826288.383783:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826288.383842:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826288.384413:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332543360 00000100:00100000:2.0:1787826288.384429:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332543360 00010000:00010000:2.0:1787826288.384494:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 7 pid: 11629 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826288.384506:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332543232 00000100:00100000:3.0:1787826288.384513:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332543232 00000100:00100000:2.0:1787826288.384557:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21872 00000100:00100000:2.0:1787826288.384569:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332543360:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826288.384589:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826288.384598:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 7 pid: 11629 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.384612:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 7 pid: 11629 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.384640:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49be4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080109e expref: 7 pid: 11629 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.384662:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f7961614400) 00010000:00010000:2.0:1787826288.384672:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49c7e expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.384691:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49c7e expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.384701:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49c7e expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.384748:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49c7e expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826288.384765:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6512:1874663346139264:0@lo:105: 00000100:00100000:3.0:1787826288.384849:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30791 00000100:00100000:2.0:1787826288.384850:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.384866:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332543232:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826288.384888:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1787826288.384903:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453d9900 going to sleep for 11 seconds 00010000:00010000:3.0:1787826288.384911:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f49c9a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010d6 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826288.384925:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346139264 00000100:00100000:2.0:1787826288.384930:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346139264 00000100:00100000:2.0:1787826288.384960:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7824 00000100:00100000:3.0:1787826288.384969:0:24218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:24218:1874663346139392:192.168.203.56@tcp:106: 00000100:00100000:2.0:1787826288.384970:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6512:x1874663346139264:12345-0@lo:105: 00010000:00010000:2.0:1787826288.384988:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 5/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.384999:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 5/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.385008:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 5/0,1 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.385016:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.385025:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.385036:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826288.385045:0:24218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bc00 going to sleep for 11 seconds 00010000:00010000:2.0:1787826288.385057:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68e00) 00000100:00100000:2.0:1787826288.385096:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6512:x1874663346139264:12345-0@lo:105: Request processed in 129us (249us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.385156:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7824 00000100:00100000:2.0:1787826288.385198:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6512:1874663346139264:0@lo:105: 00010000:00010000:2.0:1787826288.385216:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826288.385235:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332543360:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 669us (1393us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.385243:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21872 00000100:00080000:0.0:1787826288.385441:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787826288.385457:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.385469:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826288.385502:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x190:0x0] 00010000:00010000:0.0:1787826288.385543:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.385551:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.385561:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826288.385577:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346139520:0@lo:101:mdt_io00_003.0 00000100:00100000:0.0:1787826288.385610:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826288.385639:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826288.385690:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346139520 00000100:00100000:1.0:1787826288.385696:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346139520 00000100:00100000:1.0:1787826288.385736:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23389 00000100:00100000:1.0:1787826288.385744:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346139520:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:1.0:1787826288.385756:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826288.385772:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f49ca8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49ca1 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.385792:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f49ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49ca1 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.385805:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f49ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49ca1 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.385824:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x4ec94fd6 (ffff9f7961614800) 00010000:00010000:1.0:1787826288.385827:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826288.385830:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826288.385869:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346139520:12345-0@lo:101:mdt_io00_003.0 Request processed in 126us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.385875:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23389 00000100:00100000:2.0:1787826288.387182:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346139520:0@lo:101:mdt_io00_003.0 00010000:00010000:2.0:1787826288.387211:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ca8 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.387227:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ca8 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826288.387249:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x190:0x0] 00010000:00010000:2.0:1787826288.387291:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.387310:0:125380:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.387319:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.387353:0:125380:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.387368:0:125380:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 10 pid: 24702 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826288.387383:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346139648:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826288.387472:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a300 going to sleep for 11 seconds 00002000:00010000:3.0:1787826288.392246:0:24218:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000401:0x51d:0x0] updating lvb size: 0 -> 6 00002000:00010000:3.0:1787826288.392261:0:24218:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000401:0x51d:0x0] updating lvb mtime: 0 -> 1787826283 00002000:00010000:3.0:1787826288.392264:0:24218:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000401:0x51d:0x0] updating lvb atime: 0 -> 1787826283 00002000:00010000:3.0:1787826288.392266:0:24218:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000401:0x51d:0x0] updating lvb ctime: 0 -> 1787826283 00002000:00010000:3.0:1787826288.392267:0:24218:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000401:0x51d:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:3.0:1787826288.392296:0:24218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:24218:1874663346139392:192.168.203.56@tcp:106: 00010000:00010000:3.0:1787826288.392361:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f49c9a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010d6 expref: 6 pid: 24218 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.392402:0:24218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f49c9a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010d6 expref: 6 pid: 24218 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.392420:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826288.392518:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332543232:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7655us (8748us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826288.392529:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30791 00000100:00100000:0.0:1787826288.398042:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826288.398168:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332543488 00000100:00100000:1.0:1787826288.398181:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332543488 00010000:00010000:1.0:1787826288.398235:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 11 pid: 24702 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826288.398296:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21873 00000100:00100000:1.0:1787826288.398310:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332543488:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826288.398330:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826288.398339:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 11 pid: 24702 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.398354:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 11 pid: 24702 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.398384:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:1.0:1787826288.398391:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.398403:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.398411:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.398435:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826288.398748:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332543488:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 438us (718us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.398758:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21873 00010000:00010000:2.0:1787826288.399232:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49bf2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010a5 expref: 10 pid: 24702 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826288.399257:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346139648:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826288.399520:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.400397:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.400414:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.400450:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.400465:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.400477:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826288.400587:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346139776:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.400674:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.400721:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.400855:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346139776 00000100:00100000:2.0:1787826288.400865:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346139776 00000100:00100000:2.0:1787826288.400930:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23390 00000100:00100000:2.0:1787826288.400941:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346139776:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.401128:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346139776:12345-0@lo:1000:mdt_io00_003.0 Request processed in 187us (453us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.401137:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23390 00000100:00100000:1.0:1787826288.407205:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826288.407432:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332543616 00000100:00100000:2.0:1787826288.407446:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332543616 00000100:00100000:2.0:1787826288.407543:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30792 00000100:00100000:2.0:1787826288.407556:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224014:x1874663332543616:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826288.407583:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826288.407610:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010dd expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.407656:0:35096:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 5 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.407667:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x51d:0x0].0x0 since=4s v=(0 0) 00010000:00010000:2.0:1787826288.407672:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826288.407678:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 4/0,0 mode: --/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.407728:0:35096:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 5 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.407745:0:35096:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 5 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826288.407762:0:35096:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:35096:1874663346139904:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826288.407880:0:35096:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826288.409415:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346139776:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.409537:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140032:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.409614:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826288.409657:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826288.409745:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346140032 00000100:00100000:1.0:1787826288.409755:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346140032 00000100:00100000:1.0:1787826288.409827:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23391 00000100:00100000:1.0:1787826288.409838:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140032:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.409959:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140032:12345-0@lo:1000:mdt_io00_003.0 Request processed in 121us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.409968:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23391 00000100:00100000:3.0:1787826288.414427:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140032:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.414563:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848300700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140160:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.414649:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.414695:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.414714:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346140160 00000100:00100000:2.0:1787826288.414725:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346140160 00000100:00100000:2.0:1787826288.414795:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23392 00000100:00100000:2.0:1787826288.414815:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140160:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.414989:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140160:12345-0@lo:1000:mdt_io00_003.0 Request processed in 183us (339us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826288.414997:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23392 00000100:00100000:0.0:1787826288.415154:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848300700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140160:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.415246:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140288:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.420803:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826288.421001:0:35096:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:35096:1874663346139904:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826288.421049:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 3/0,0 mode: --/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.421076:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826288.421183:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224014:x1874663332543616:12345-192.168.203.56@tcp:101:cat.0 Request processed in 13629us (13992us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.421193:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30792 00000100:00100000:1.0:1787826288.421530:0:24159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332543744 00000100:00100000:1.0:1787826288.421542:0:24159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332543744 00002000:00100000:1.0:1787826288.421605:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000401:0x51d:0x0] (0->5) req@ffff9f7848300700 x1874663332543744/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:644/0 lens 496/0 e 0 to 0 dl 1787826299 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826288.421617:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7848300700 with x1874663332543744 ext(0->5) 00010000:00010000:1.0:1787826288.421627:0:24159:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 6 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826288.421645:0:24159:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 6 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826288.421654:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9f7848300700 00000100:00100000:1.0:1787826288.421691:0:24159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2489 00000100:00100000:1.0:1787826288.421702:0:24159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332543744:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:0.0:1787826288.423673:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826288.423730:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826288.424673:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346140288 00000100:00100000:1.0:1787826288.424690:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346140288 00000100:00100000:1.0:1787826288.424775:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23393 00000100:00100000:1.0:1787826288.424788:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140288:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.424968:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140288:12345-0@lo:1000:mdt_io00_003.0 Request processed in 181us (1304us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.424977:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23393 00000100:00100000:3.0:1787826288.425156:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140288:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.425294:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140416:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.425343:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826288.425404:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346140416 00000100:00100000:2.0:1787826288.425411:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346140416 00000100:00100000:2.0:1787826288.425453:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23394 00000100:00100000:2.0:1787826288.425462:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140416:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.425568:0:24159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332543744:12345-192.168.203.56@tcp:4:bash.0 Request processed in 3864us (4776us total) trans 4294973839 rc 0/0 00000100:00100000:0.0:1787826288.425579:0:24159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2489 00002000:00100000:0.0:1787826288.425590:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000401:0x51d:0x0] (0->5) req@ffff9f7848300700 x1874663332543744/t4294973839(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:644/0 lens 496/448 e 0 to 0 dl 1787826299 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826288.425601:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7848300700 with x1874663332543744 ext(0->5) 00010000:00010000:0.0:1787826288.425614:0:24159:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 8 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.425634:0:24159:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 8 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826288.425643:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9f7848300700 00000100:00100000:3.0:1787826288.427452:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826288.428526:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826288.430473:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332543872 00000100:00100000:2.0:1787826288.430493:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332543872 00010000:00010000:2.0:1787826288.430562:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 8 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826288.430623:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21874 00000100:00100000:2.0:1787826288.430636:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332543872:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826288.430657:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1787826288.430681:0:125406:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x51d:0x0] updating lvb blocks from disk: 1 -> 8 00010000:00010000:2.0:1787826288.430690:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 8 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.430706:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 8 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.430736:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f49b6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801074 expref: 8 pid: 35097 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.430756:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826288.430761:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826288.430772:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.430783:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.430839:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826288.430856:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:125406:1874663346140544:192.168.203.56@tcp:105: 00000100:00100000:2.0:1787826288.430953:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.431033:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140416:12345-0@lo:1000:mdt_io00_003.0 Request processed in 5570us (5687us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.431047:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23394 00000100:00100000:3.0:1787826288.431237:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140416:0@lo:1000:mdt_io00_003.0 00000004:00080000:3.0:1787826288.431292:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e44:0x0] 00010000:00010000:3.0:1787826288.431409:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.431426:0:125380:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.431435:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.431460:0:125380:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.431475:0:125380:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 9 pid: 24703 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826288.431488:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346140672:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826288.431538:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.442491:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:125406:1874663346140544:192.168.203.56@tcp:105: 00010000:00010000:2.0:1787826288.442527:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826288.442613:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332543872:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 11979us (14098us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.442626:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21874 00000100:00100000:0.0:1787826288.443676:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346140672:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826288.443714:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826288.454408:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826288.454587:0:20179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332544000 00000100:00100000:2.0:1787826288.454600:0:20179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332544000 00002000:00100000:2.0:1787826288.454667:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x51d:0x0] (0->4095) req@ffff9f79785e5500 x1874663332544000/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:644/0 lens 488/0 e 0 to 0 dl 1787826299 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826288.454680:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f79785e5500 with x1874663332544000 ext(0->4095) 00002000:00010000:2.0:1787826288.454689:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f79785e5500 00000100:00100000:2.0:1787826288.454736:0:20179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27634 00000100:00100000:2.0:1787826288.454748:0:20179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332544000:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:2.0:1787826288.454978:0:20179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332544000:12345-192.168.203.56@tcp:3:cat.0 Request processed in 231us (585us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787826288.454987:0:20179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27634 00002000:00100000:2.0:1787826288.454994:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x51d:0x0] (0->4095) req@ffff9f79785e5500 x1874663332544000/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:644/0 lens 488/4536 e 0 to 0 dl 1787826299 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826288.455003:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f79785e5500 with x1874663332544000 ext(0->4095) 00002000:00010000:2.0:1787826288.455007:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f79785e5500 00000100:00100000:3.0:1787826288.457725:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826288.457844:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332544128 00000100:00100000:2.0:1787826288.457862:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332544128 00010000:00010000:2.0:1787826288.457919:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 10 pid: 24703 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826288.457974:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21875 00000100:00100000:2.0:1787826288.457987:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2396:x1874663332544128:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826288.458008:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826288.458017:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 10 pid: 24703 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.458031:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 10 pid: 24703 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.458059:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49c8c lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ac expref: 10 pid: 24703 timeout: 8200 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.458081:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f7961614b00) 00010000:00010000:2.0:1787826288.458087:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.458098:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.458158:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.458188:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826288.458250:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332544128:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 265us (538us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.458258:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21875 00010000:00010000:0.0:1787826288.458427:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.458437:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.458508:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.458517:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.458528:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826288.458552:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140800:0@lo:101:mdt_io00_003.0 00000100:00100000:0.0:1787826288.458605:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826288.458646:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826288.458719:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826288.458849:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332544256 00000100:00100000:0.0:1787826288.458857:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332544256 00000100:00100000:0.0:1787826288.458909:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68282 00000100:00100000:0.0:1787826288.458919:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224014:x1874663332544256:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826288.458944:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826288.458958:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49cd2 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1db6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010e4 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.459013:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49cd2 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1db6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010e4 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.459026:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49cd2 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1db6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010e4 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.459047:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f7961614b00) 00010000:00010000:0.0:1787826288.459051:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787826288.459054:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826288.459087:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224014:x1874663332544256:12345-192.168.203.56@tcp:101:cat.0 Request processed in 170us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826288.459094:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68282 00000100:00100000:2.0:1787826288.459569:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346140800 00000100:00100000:2.0:1787826288.459577:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346140800 00000100:00100000:2.0:1787826288.459625:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23395 00000100:00100000:2.0:1787826288.459634:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346140800:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:2.0:1787826288.459646:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826288.459667:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49cd9 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49ccb expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.459687:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49cd9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49ccb expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.459701:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49cd9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49ccb expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826288.459709:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7952ca9900) 00010000:00010000:2.0:1787826288.459712:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787826288.459714:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826288.459770:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346140800:12345-0@lo:101:mdt_io00_003.0 Request processed in 138us (1164us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.459777:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23395 00000100:00100000:3.0:1787826288.459961:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140800:0@lo:101:mdt_io00_003.0 00010000:00010000:3.0:1787826288.459983:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49cd9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.459997:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49cd9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826288.460025:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e45:0x0] 00000100:00100000:3.0:1787826288.460098:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140928:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.460161:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.460190:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826288.460635:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346140928 00000100:00100000:0.0:1787826288.460643:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346140928 00000100:00100000:0.0:1787826288.460688:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23396 00000100:00100000:0.0:1787826288.460699:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346140928:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.460814:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346140928:12345-0@lo:1000:mdt_io00_003.0 Request processed in 108us (644us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826288.460823:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23396 00000100:00100000:3.0:1787826288.460941:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346140928:0@lo:1000:mdt_io00_003.0 00000004:00080000:3.0:1787826288.460978:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x190:0x0] 00000100:00100000:3.0:1787826288.461022:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141056:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.461052:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826288.461375:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346141056 00000100:00100000:1.0:1787826288.461382:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346141056 00000100:00100000:1.0:1787826288.461427:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23397 00000100:00100000:1.0:1787826288.461437:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141056:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.461519:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141056:12345-0@lo:1000:mdt_io00_003.0 Request processed in 84us (465us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826288.461526:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23397 00000100:00100000:3.0:1787826288.461829:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826288.461850:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141056:0@lo:1000:mdt_io00_003.0 00000004:00080000:3.0:1787826288.461878:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e44:0x0] 00010000:00010000:3.0:1787826288.461930:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.461947:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826288.461958:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826288.462012:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141184:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.462048:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.462084:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826288.462177:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346141184 00000100:00100000:0.0:1787826288.462184:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346141184 00000100:00100000:0.0:1787826288.462225:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23398 00000100:00100000:0.0:1787826288.462234:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141184:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826288.462615:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141184:12345-0@lo:1000:mdt_io00_003.0 Request processed in 380us (564us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826288.462623:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23398 00000100:00100000:3.0:1787826288.462649:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826288.462711:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141184:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826288.462738:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e44:0x0] 00000100:00100000:1.0:1787826288.462772:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141312:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.462803:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826288.462841:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.462892:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346141312 00000100:00100000:2.0:1787826288.462898:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346141312 00000100:00100000:2.0:1787826288.462939:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23399 00000100:00100000:2.0:1787826288.462948:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141312:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.463023:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141312:12345-0@lo:1000:mdt_io00_003.0 Request processed in 76us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.463030:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23399 00000100:00100000:0.0:1787826288.463090:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332544512 00000100:00100000:0.0:1787826288.463094:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332544512 00000100:00100000:0.0:1787826288.463162:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9312 00000100:00100000:0.0:1787826288.463172:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd2c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224014:x1874663332544512:12345-192.168.203.56@tcp:35:cat.0 00000100:00100000:1.0:1787826288.463550:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141312:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.463598:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141440:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.463627:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826288.463653:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.463709:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346141440 00000100:00100000:2.0:1787826288.463714:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346141440 00000100:00100000:2.0:1787826288.463751:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23400 00000100:00100000:2.0:1787826288.463759:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141440:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.463841:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141440:12345-0@lo:1000:mdt_io00_003.0 Request processed in 83us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.463849:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23400 00000100:00100000:3.0:1787826288.464178:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141440:0@lo:1000:mdt_io00_003.0 00000004:00080000:3.0:1787826288.464207:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x190:0x0] 00000100:00100000:3.0:1787826288.464246:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850359500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141568:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.464280:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.464308:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753ba80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826288.464893:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd2c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224014:x1874663332544512:12345-192.168.203.56@tcp:35:cat.0 Request processed in 1718us (2242us total) trans 8589957335 rc 0/0 00000100:00100000:0.0:1787826288.464904:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9312 00000100:00100000:2.0:1787826288.465380:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346141568 00000100:00100000:2.0:1787826288.465391:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346141568 00000100:00100000:2.0:1787826288.465444:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23401 00000100:00100000:2.0:1787826288.465454:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141568:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.465574:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141568:12345-0@lo:1000:mdt_io00_003.0 Request processed in 121us (1293us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.465582:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23401 00000100:00100000:1.0:1787826288.466529:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850359500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141568:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826288.466584:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e44:0x0] 00000100:00100000:1.0:1787826288.466664:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141696:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826288.466724:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826288.466766:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.468441:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346141696 00000100:00100000:2.0:1787826288.468458:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346141696 00000100:00100000:2.0:1787826288.468558:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23402 00000100:00100000:2.0:1787826288.468571:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141696:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.468734:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141696:12345-0@lo:1000:mdt_io00_003.0 Request processed in 166us (2009us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.468742:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23402 00000100:00100000:3.0:1787826288.469180:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141696:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.469265:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141824:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.469313:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.469414:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.476569:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346141824 00000100:00100000:2.0:1787826288.476594:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346141824 00000100:00100000:2.0:1787826288.476678:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23403 00000100:00100000:2.0:1787826288.476689:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141824:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.476887:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141824:12345-0@lo:1000:mdt_io00_003.0 Request processed in 198us (7572us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.476895:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23403 00000100:00100000:3.0:1787826288.478265:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141824:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.478622:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141952:0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826288.478697:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826288.478743:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826288.479394:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346141952 00000100:00100000:2.0:1787826288.479405:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346141952 00000100:00100000:2.0:1787826288.479528:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23404 00000100:00100000:2.0:1787826288.479541:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141952:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826288.479691:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346141952:12345-0@lo:1000:mdt_io00_003.0 Request processed in 153us (994us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826288.479700:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23404 00000100:00100000:3.0:1787826288.479788:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346141952:0@lo:1000:mdt_io00_003.0 00000040:00080000:3.0:1787826288.484494:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].211, 64 off21696 00000004:00080000:3.0:1787826288.484647:0:125380:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8100696539957ns, next at 8101696416951ns, rc = 0 00000040:00080000:3.0:1787826288.484752:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].931, 904 off876056 00000040:00080000:3.0:1787826288.484857:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].934, 984 off974880 00000004:00080000:3.0:1787826288.484865:0:125380:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784753b6c0 with version 1816 00000004:00080000:0.0:1787826288.484964:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1816 rpc_version 1816 00000020:00100000:0.0:1787826288.485013:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826288.485027:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7849dd0e00 x1874663346142208/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2584/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826288.485060:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346142208:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826288.485230:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826288.492485:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826288.495405:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346142208 00000100:00100000:1.0:1787826288.495428:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346142208 00000100:00100000:0.0:1787826288.496550:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23405 00000100:00100000:0.0:1787826288.496568:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346142208:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826288.497280:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346142208:12345-0@lo:1000:osp_up1-0.0 Request processed in 714us (12048us total) trans 4294985348 rc 0/0 00000100:00100000:0.0:1787826288.497290:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23405 00000100:00100000:2.0:1787826288.497666:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346142208:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787826288.499377:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499397:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499406:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499413:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826288.499418:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499426:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499446:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f7961614b00) 00010000:00010000:0.0:1787826288.499449:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826288.499455:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499461:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f49cc4 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826288.499485:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e45:0x0] 00010000:00010000:0.0:1787826288.499495:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49cd9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499504:0:125380:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49cd9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499518:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499540:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499547:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499553:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826288.499557:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499563:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499572:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:0.0:1787826288.499575:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826288.499579:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499585:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f49caf lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826288.499609:0:125380:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000031f69305 reply state 00000000ecb73c3e transno 8589957336 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499625:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499633:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f784abfe700) 00010000:00010000:0.0:1787826288.499635:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826288.499639:0:125380:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:0.0:1787826288.499644:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x190:0x0] 00010000:00010000:0.0:1787826288.499648:0:125380:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499655:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499661:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499695:0:125380:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826288.499701:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x190:0x0] 00010000:00010000:0.0:1787826288.499706:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ca8 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499714:0:125380:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ca8 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499724:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499730:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499735:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499740:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826288.499743:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499753:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499761:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f79460a2600) 00010000:00010000:0.0:1787826288.499763:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826288.499767:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499773:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f49ce0 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499819:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499831:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499841:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499848:0:125380:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499854:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499859:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.499883:0:125380:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1787826288.500000:0:125380:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224016:x1874663332543104:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 158125us (158363us total) trans 8589957336 rc 0/0 00000100:00100000:0.0:1787826288.500010:0:125380:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1225 00010000:00010000:0.0:1787826288.500395:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826288.500404:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500412:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826288.500417:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826288.500427:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500472:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 2/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500479:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826288.500513:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 1/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500522:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68e00/0xe21309aa81f49c7e lrc: 0/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49c85 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500548:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826288.500552:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500558:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826288.500561:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500569:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500578:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f7961614b00) 00010000:00010000:0.0:1787826288.500581:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826288.500584:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500590:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f49ce7 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.500600:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826288.500602:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826288.500604:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826288.500652:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826288.500678:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346142336:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826288.500754:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826288.501008:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826288.502820:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346142336 00000100:00100000:0.0:1787826288.502842:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346142336 00000100:00100000:0.0:1787826288.502930:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11023 00000100:00100000:0.0:1787826288.502944:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c81c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+9:3652:x1874663346142336:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826288.502966:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826288.502998:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f49c85 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49c7e expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826288.503022:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f7961614400) 00010000:00010000:0.0:1787826288.503040:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826288.503205:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c81c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346142336:12345-0@lo:103:ldlm_bl.0 Request processed in 264us (2450us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826288.503214:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11023 00000100:00100000:1.0:1787826288.503241:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346142336:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826288.812953:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826288.862514:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826288.862586:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826288.862627:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826288.862678:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332544640 00000100:00100000:3.0:1787826288.862690:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332544640 00000100:00100000:3.0:1787826288.862757:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2212 00000100:00100000:3.0:1787826288.862771:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf42c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332544640:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826288.862874:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf42c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332544640:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 105us (287us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826288.862883:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2212 00000100:00100000:3.0:1787826288.863075:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332544768 00000100:00100000:3.0:1787826288.863085:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332544768 00000100:00100000:3.0:1787826288.863175:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332544896 00000100:00100000:3.0:1787826288.863179:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332544896 00000100:00100000:3.0:1787826288.863203:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8777 00000100:00100000:3.0:1787826288.863210:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332544768:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826288.863256:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332544768:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 45us (773us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826288.863263:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8777 00000100:00100000:3.0:1787826288.863276:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8778 00000100:00100000:3.0:1787826288.863284:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332544896:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826288.863310:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332544896:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (682us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826288.863316:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8778 00000100:00080000:0.0:1787826288.863662:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787826289.264733:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49ca8 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826289.264761:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49ca8 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.264821:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.264831:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.264839:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.264846:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826289.264851:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.264860:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.264883:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f784abfe700) 00010000:00010000:2.0:1787826289.264887:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826289.264892:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.264903:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49b58 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265004:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49ca8 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265012:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826289.265038:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42200/0xe21309aa81f49ca1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49ca8 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265060:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49cd9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826289.265068:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49cd9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265086:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49cd9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265093:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826289.265100:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43400/0xe21309aa81f49ccb lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49cd9 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826289.265147:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957336 is committed 00000004:00080000:3.0:1787826289.265154:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 210 committed 00000001:00080000:3.0:1787826289.265172:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957335 is committed 00000020:00080000:3.0:1787826289.265185:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979077 batchid updated 00000001:00080000:3.0:1787826289.265196:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957333 is committed 00010000:00010000:3.0:1787826289.265225:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265232:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265239:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265245:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826289.265248:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265255:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265265:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f784abfe700) 00010000:00010000:3.0:1787826289.265268:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826289.265271:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.265277:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f49cb6 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826289.265614:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826289.265637:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346142720:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1787826289.265730:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826289.265792:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346142720 00000100:00100000:0.0:1787826289.265812:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346142720 00000100:00100000:0.0:1787826289.265881:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30793 00000100:00100000:0.0:1787826289.265894:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346142720:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1787826289.265928:0:17826:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1309 count 1 00010000:00010000:0.0:1787826289.265958:0:17826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.265987:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.265996:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=1s v=(1 0) 00010000:00010000:0.0:1787826289.266001:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826289.266005:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 4/0,1 mode: --/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.266048:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.266065:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 5 pid: 35096 timeout: 8201 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826289.266077:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346142848:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826289.266172:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f3c0 going to sleep for 11 seconds 00000100:00080000:0.0:1787826289.266186:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826289.267159:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850359500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346142464:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826289.267222:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826289.267260:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787826289.267268:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346142592:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826289.267289:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:2.0:1787826289.267320:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979077 00000004:00080000:2.0:1787826289.267403:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784548a300 with version 1817 00000020:00080000:2.0:1787826289.267413:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979077 cancel update log [0x240000404:0x2:0x0].933: rc = 0 00000020:00080000:2.0:1787826289.269643:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979077 cancel update log [0x200000bd0:0x2:0x0].930: rc = 0 00000100:00100000:3.0:1787826289.270019:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346142464 00000100:00100000:3.0:1787826289.270033:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346142464 00000100:00100000:3.0:1787826289.270137:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346142592 00000100:00100000:3.0:1787826289.270142:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346142592 00000100:00100000:3.0:1787826289.270176:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11024 00000100:00100000:3.0:1787826289.270189:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346142464:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826289.270209:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826289.270241:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f49ca8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x190:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49ca1 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.270263:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x4ec94fd6 (ffff9f7961614800) 00010000:00010000:3.0:1787826289.270277:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826289.270376:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346142464:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 189us (3153us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.270385:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11024 00000100:00100000:3.0:1787826289.270408:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11025 00000100:00100000:3.0:1787826289.270415:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346142592:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826289.270424:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826289.270482:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49cd9 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49ccb expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.270511:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7952ca9900) 00010000:00010000:3.0:1787826289.270520:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826289.270546:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346142592:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 132us (3255us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.270552:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11025 00000004:00080000:3.0:1787826289.270596:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1817 rpc_version 1817 00000020:00100000:3.0:1787826289.270634:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826289.270644:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7847b5bb80 x1874663346143104/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000020:00080000:2.0:1787826289.270987:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979077 00000100:00100000:2.0:1787826289.271039:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346142592:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826289.271051:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850359500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346142464:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826289.276420:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346143104:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826289.276603:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.276659:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.276690:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346143104 00000100:00100000:2.0:1787826289.276702:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346143104 00000100:00100000:2.0:1787826289.276785:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23406 00000100:00100000:2.0:1787826289.276797:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346143104:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826289.277046:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346143104:12345-0@lo:1000:osp_up1-0.0 Request processed in 250us (442us total) trans 4294985349 rc 0/0 00000100:00100000:2.0:1787826289.277055:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23406 00000100:00100000:3.0:1787826289.277935:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.277981:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332545024 00000100:00100000:1.0:1787826289.277993:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332545024 00000100:00100000:3.0:1787826289.278043:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346142848:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826289.278045:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 6 pid: 35096 timeout: 8201 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.278099:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21876 00000100:00100000:1.0:1787826289.278140:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332545024:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826289.278159:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826289.278167:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 6 pid: 35096 timeout: 8201 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.278182:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 6 pid: 35096 timeout: 8201 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.278225:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064494fc1/0xe21309aa81f49cbd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608010dd expref: 6 pid: 35096 timeout: 8201 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.278246:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826289.278252:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826289.278259:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.278277:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.278305:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826289.278378:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332545024:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 268us (454us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.278386:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21876 00010000:00010000:3.0:1787826289.279663:0:17826:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.279698:0:17826:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.279707:0:17826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.279720:0:17826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.279730:0:17826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.279759:0:17826:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826289.280036:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346142720:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14141us (14303us total) trans 4294973840 rc 0/0 00000100:00100000:3.0:1787826289.280048:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30793 00000100:00100000:3.0:1787826289.280170:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346143104:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787826289.280224:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7849dd0380/2, rc 0, transno 4294973840 00000100:00100000:2.0:1787826289.280233:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346142720:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1787826289.297061:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826289.297087:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.297164:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826289.297172:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.297187:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.297218:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 1/0,0 mode: --/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.297226:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f49cee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.297252:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826289.297256:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826289.297258:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826289.380455:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826289.380585:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332545280 00000100:00100000:2.0:1787826289.380599:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332545280 00000100:00100000:2.0:1787826289.380682:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30794 00000100:00100000:2.0:1787826289.380696:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332545280:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:2.0:1787826289.380870:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332545280:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 175us (429us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826289.380879:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30794 00000100:00100000:2.0:1787826289.381872:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.381952:0:8449:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332545152 00000100:00100000:1.0:1787826289.381962:0:8449:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332545152 00002000:00100000:1.0:1787826289.382016:0:8449:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x4c8:0x0] (0->5) req@ffff9f7850359500 x1874663332545152/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:645/0 lens 496/0 e 0 to 0 dl 1787826300 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826289.382029:0:8449:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7850359500 with x1874663332545152 ext(0->5) 00002000:00010000:1.0:1787826289.382036:0:8449:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7850359500 00000100:00100000:1.0:1787826289.382069:0:8449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27635 00000100:00100000:1.0:1787826289.382080:0:8449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850359500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332545152:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826289.383847:0:8449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850359500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332545152:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1757us (1971us total) trans 4294976148 rc 0/0 00000100:00100000:2.0:1787826289.383863:0:8449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27635 00002000:00100000:2.0:1787826289.383878:0:8449:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x4c8:0x0] (0->5) req@ffff9f7850359500 x1874663332545152/t4294976148(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:645/0 lens 496/448 e 0 to 0 dl 1787826300 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826289.383888:0:8449:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7850359500 with x1874663332545152 ext(0->5) 00002000:00010000:2.0:1787826289.383894:0:8449:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7850359500 00000100:00100000:3.0:1787826289.651813:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826289.651845:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346143232:0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.651959:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.651971:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346143360:0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.652021:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346143616:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.652031:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.652063:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.652072:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346143872:0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.652095:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346144128:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.652141:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.652163:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.652188:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1787826289.652193:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346143744:0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.652216:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.652233:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346144256:0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.652255:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.652290:0:30904:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346143232 00000100:00100000:3.0:1787826289.652300:0:30904:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346143232 00000100:00100000:3.0:1787826289.652374:0:30904:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2213 00000100:00100000:3.0:1787826289.652387:0:30904:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf42850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346143232:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.652453:0:30904:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf42850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346143232:12345-0@lo:400:kworker.0 Request processed in 67us (495us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.652461:0:30904:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2213 00000100:00100000:1.0:1787826289.652580:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850773480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346144384:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.652609:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.652635:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1787826289.652641:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346143488:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.652665:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.652693:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346144000:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.652723:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.652741:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346144512:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.652763:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.652796:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346143616 00000100:00100000:3.0:1787826289.652812:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346143616 00000100:00100000:1.0:1787826289.652816:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346143360 00000100:00100000:1.0:1787826289.652824:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346143360 00000100:00100000:3.0:1787826289.652866:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346144128 00000100:00100000:3.0:1787826289.652871:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346144128 00000100:00100000:1.0:1787826289.652875:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23407 00000100:00100000:1.0:1787826289.652887:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346143360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.652905:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346143744 00000100:00100000:3.0:1787826289.652910:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346143744 00000100:00100000:3.0:1787826289.652940:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346143488 00000100:00100000:3.0:1787826289.652944:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346143488 00000100:00100000:1.0:1787826289.652947:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346143360:12345-0@lo:400:kworker.0 Request processed in 63us (915us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.652956:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23407 00000100:00100000:3.0:1787826289.652974:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346144000 00000100:00100000:3.0:1787826289.652979:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346144000 00000100:00100000:1.0:1787826289.652986:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346143872 00000100:00100000:1.0:1787826289.652993:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346143872 00000100:00100000:3.0:1787826289.653007:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346144512 00000100:00100000:3.0:1787826289.653013:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346144512 00000100:00100000:1.0:1787826289.653032:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346144256 00000100:00100000:1.0:1787826289.653037:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346144256 00000100:00100000:3.0:1787826289.653047:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14249 00000100:00100000:3.0:1787826289.653057:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346143616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.653064:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346144384 00000100:00100000:1.0:1787826289.653068:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346144384 00000100:00100000:1.0:1787826289.653096:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8779 00000100:00100000:3.0:1787826289.653102:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346143616:12345-0@lo:400:kworker.0 Request processed in 48us (1039us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.653137:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14249 00000100:00100000:3.0:1787826289.653152:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14250 00000100:00100000:3.0:1787826289.653159:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346144128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.653191:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346144128:12345-0@lo:400:kworker.0 Request processed in 34us (1029us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.653196:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14250 00000100:00100000:3.0:1787826289.653210:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14251 00000100:00100000:3.0:1787826289.653215:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346143744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.653239:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346143744:12345-0@lo:400:kworker.0 Request processed in 24us (1022us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.653244:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14251 00000100:00100000:3.0:1787826289.653256:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14252 00000100:00100000:3.0:1787826289.653261:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346143488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.653287:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346143488:12345-0@lo:400:kworker.0 Request processed in 25us (621us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.653293:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14252 00000100:00100000:3.0:1787826289.653304:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14253 00000100:00100000:3.0:1787826289.653310:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346144000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.653408:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346144000:12345-0@lo:400:kworker.0 Request processed in 98us (684us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.653414:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14253 00000100:00100000:3.0:1787826289.653429:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14254 00000100:00100000:3.0:1787826289.653435:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346144512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.653459:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346144512:12345-0@lo:400:kworker.0 Request processed in 25us (696us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.653465:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14254 00000100:00100000:1.0:1787826289.654347:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346143872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.654417:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346143872:12345-0@lo:400:kworker.0 Request processed in 90us (2277us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.654424:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8779 00000100:00100000:1.0:1787826289.654453:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8780 00000100:00100000:1.0:1787826289.654461:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346144256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.654494:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346144256:12345-0@lo:400:kworker.0 Request processed in 34us (2237us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.654499:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8780 00000100:00100000:1.0:1787826289.654509:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8781 00000100:00100000:1.0:1787826289.654516:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346144384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.654546:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346144384:12345-0@lo:400:kworker.0 Request processed in 31us (1937us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.654552:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8781 00000100:00100000:1.0:1787826289.654633:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346143488:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.654649:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346144000:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.654660:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346144512:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.654672:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957333 gen 2 00000004:00080000:1.0:1787826289.654677:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957330 committed transno 8589957333 00000004:00080000:1.0:1787826289.654704:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957331 committed transno 8589957333 00000004:00080000:1.0:1787826289.654720:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957332 committed transno 8589957333 00000004:00080000:1.0:1787826289.654738:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957333 committed transno 8589957333 00000100:00100000:1.0:1787826289.654753:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346143360:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.654767:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346143872:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.654777:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976147 gen 2 00000004:00080000:1.0:1787826289.654781:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7845aff100, transno 4294976147 00000100:00100000:1.0:1787826289.654812:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850773480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346144384:0@lo:400:kworker.0 00000100:00080000:1.0:1787826289.657562:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826289.657577:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826289.657582:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826289.657587:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826289.657693:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346143744:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.657712:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976145 gen 2 00000004:00080000:1.0:1787826289.657718:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784d98c380, transno 4294976145 00000100:00100000:1.0:1787826289.657745:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346144256:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.657761:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346143616:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.657773:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346144128:0@lo:400:kworker.0 00000100:00100000:1.0:1787826289.657790:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346143232:0@lo:400:kworker.0 00000100:00100000:3.0:1787826289.802746:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.804079:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332545408 00000100:00100000:1.0:1787826289.804095:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332545408 00000100:00100000:1.0:1787826289.804218:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68283 00000100:00100000:1.0:1787826289.804231:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224052:x1874663332545408:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826289.804258:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826289.804287:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49cf5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010eb expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.804375:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49cfc lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.804393:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49cfc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.804410:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49cfc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826289.804550:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e44:0x0]: returning size 4096 00000004:00200000:1.0:1787826289.804556:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787826289.804581:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49cfc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.804618:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f49cf5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010eb expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.804644:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49cfc lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010eb expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.804660:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7961614600) 00010000:00010000:1.0:1787826289.804664:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826289.804669:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:1.0:1787826289.804759:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224052:x1874663332545408:12345-192.168.203.56@tcp:101:rm.0 Request processed in 530us (2028us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.804767:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68283 00000100:00080000:0.0:1787826289.806455:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826289.808386:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.808469:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332545536 00000100:00100000:1.0:1787826289.808491:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332545536 00000100:00100000:1.0:1787826289.808575:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68284 00000100:00100000:1.0:1787826289.808591:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224052:x1874663332545536:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826289.808626:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826289.808663:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49d03 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010f2 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.808756:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.808774:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.808789:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826289.808858:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x190:0x0]: returning size 4096 00000004:00200000:1.0:1787826289.808864:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826289.808907:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49d03 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.808935:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.808945:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:1.0:1787826289.808949:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826289.808953:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826289.809172:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224052:x1874663332545536:12345-192.168.203.56@tcp:101:rm.0 Request processed in 582us (796us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.809181:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68284 00000100:00100000:0.0:1787826289.814428:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.814595:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332545664 00000100:00100000:1.0:1787826289.814614:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332545664 00000100:00100000:1.0:1787826289.814727:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68285 00000100:00100000:1.0:1787826289.814741:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224052:x1874663332545664:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826289.814772:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826289.814855:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49d11 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608010f9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.814940:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.814957:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.814973:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826289.815284:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18e:0x0]: returning size 4096 00000004:00200000:1.0:1787826289.815292:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826289.815348:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49d11 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.815368:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.815378:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961614800) 00010000:00010000:1.0:1787826289.815381:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826289.815386:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826289.815474:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224052:x1874663332545664:12345-192.168.203.56@tcp:101:rm.0 Request processed in 737us (1056us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.815485:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68285 00000100:00080000:0.0:1787826289.815659:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826289.825704:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.826363:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332545920 00000100:00100000:1.0:1787826289.826380:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332545920 00000100:00100000:1.0:1787826289.826465:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68286 00000100:00100000:1.0:1787826289.826478:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224052:x1874663332545920:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826289.826514:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826289.826537:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49d1f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801100 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826289.826685:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e44:0x0]: returning size 4096 00010000:00010000:1.0:1787826289.826783:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49d1f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801100 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.826808:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7961614000) 00010000:00010000:1.0:1787826289.826812:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826289.826817:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49d1f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801100 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.826836:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826289.826906:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224052:x1874663332545920:12345-192.168.203.56@tcp:101:rm.0 Request processed in 431us (1216us total) trans 4294985350 rc 301/301 00000100:00100000:1.0:1787826289.826913:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68286 00010000:00010000:0.0:1787826289.839460:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826289.841853:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826289.842003:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332546048 00000100:00100000:0.0:1787826289.842013:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332546048 00000100:00100000:0.0:1787826289.842469:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9313 00000100:00100000:0.0:1787826289.842480:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224052:x1874663332546048:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:0.0:1787826289.848574:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224052:x1874663332546048:12345-192.168.203.56@tcp:37:rm.0 Request processed in 6082us (6716us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826289.848588:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9313 00000100:00100000:3.0:1787826289.858945:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826289.859241:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332546176 00000100:00100000:0.0:1787826289.859255:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332546176 00000100:00100000:0.0:1787826289.859337:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9314 00000100:00100000:0.0:1787826289.859350:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224052:x1874663332546176:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826289.867719:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224052:x1874663332546176:12345-192.168.203.56@tcp:37:rm.0 Request processed in 8359us (8775us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.867736:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9314 00000100:00100000:3.0:1787826289.871604:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826289.871787:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332546432 00000100:00100000:2.0:1787826289.871811:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332546432 00000100:00100000:2.0:1787826289.871886:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9315 00000100:00100000:2.0:1787826289.871899:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224052:x1874663332546432:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:2.0:1787826289.872414:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1182, 344 off1075552 00000040:00080000:2.0:1787826289.872471:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1501, 424 off1515904 00000004:00080000:2.0:1787826289.872476:0:6530:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784aa28cc0 with version 6796 00000004:00080000:3.0:1787826289.872845:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6796 rpc_version 6796 00000020:00100000:3.0:1787826289.872887:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826289.872900:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7851242680 x1874663346144768/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826289.872928:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346144768:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826289.873005:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.873046:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.873427:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346144768 00000100:00100000:2.0:1787826289.873437:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346144768 00000100:00100000:2.0:1787826289.873496:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23408 00000100:00100000:2.0:1787826289.873503:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346144768:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826289.874006:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346144768:12345-0@lo:1000:osp_up0-1.0 Request processed in 500us (998us total) trans 8589957337 rc 0/0 00000100:00100000:2.0:1787826289.874017:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23408 00000100:00100000:3.0:1787826289.874340:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346144768:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826289.874596:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224052:x1874663332546432:12345-192.168.203.56@tcp:35:rm.0 Request processed in 2696us (3005us total) trans 4294985351 rc 0/0 00000100:00100000:2.0:1787826289.874606:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9315 00000100:00100000:3.0:1787826289.877516:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826289.878711:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332546688 00000100:00100000:2.0:1787826289.878726:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332546688 00000100:00100000:2.0:1787826289.878898:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68287 00000100:00100000:2.0:1787826289.878913:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224052:x1874663332546688:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826289.878946:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826289.878987:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49d2d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801107 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.879216:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f49d34 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.879251:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f49d34 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.879266:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f49d34 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826289.879280:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e44:0x0]: returning size 4096 00010000:00010000:2.0:1787826289.879403:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49d2d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801107 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.879433:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f49d34 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801107 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.879442:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7961614600) 00010000:00010000:2.0:1787826289.879445:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1787826289.879450:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826289.879608:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224052:x1874663332546688:12345-192.168.203.56@tcp:101:rm.0 Request processed in 698us (2107us total) trans 4294985352 rc 0/0 00000100:00100000:2.0:1787826289.879617:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68287 00000100:00100000:0.0:1787826289.887919:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.888049:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332546816 00000100:00100000:1.0:1787826289.888062:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332546816 00000100:00100000:1.0:1787826289.888331:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68288 00000100:00100000:1.0:1787826289.888345:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224052:x1874663332546816:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:1.0:1787826289.888419:0:35095:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345618944 00000100:00100000:1.0:1787826289.888517:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224052:x1874663332546816:12345-192.168.203.56@tcp:41:rm.0 Request processed in 174us (611us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.888526:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68288 00000100:00100000:1.0:1787826289.898652:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826289.898753:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332546944 00000100:00100000:2.0:1787826289.898767:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332546944 00000100:00100000:2.0:1787826289.898850:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68289 00000100:00100000:2.0:1787826289.898862:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224052:x1874663332546944:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826289.898890:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826289.898922:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d42 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080110e expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.899006:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.899307:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.899322:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.899343:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.899352:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.899359:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900416:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49d57 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900436:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49d57 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900446:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49d57 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826289.900480:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49d57 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900496:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900504:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900512:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900520:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900528:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826289.900532:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900542:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900557:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961614800) 00010000:00010000:2.0:1787826289.900561:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826289.900567:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900574:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49d49 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900589:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900596:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900602:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900608:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826289.900611:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900618:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900626:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f7952ca9b00) 00010000:00010000:2.0:1787826289.900628:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826289.900631:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900638:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d50 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900688:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d42 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080110e expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900705:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49d57 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080110e expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.900713:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7952ca9200) 00010000:00010000:2.0:1787826289.900716:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826289.900718:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826289.900831:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224052:x1874663332546944:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1969us (2193us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826289.900842:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68289 00000100:00100000:3.0:1787826289.910260:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.910782:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332547072 00000100:00100000:1.0:1787826289.910807:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332547072 00000100:00100000:1.0:1787826289.910888:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68290 00000100:00100000:1.0:1787826289.910902:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224052:x1874663332547072:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826289.910939:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826289.910971:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49d5e lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801115 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.911060:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49d65 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.911077:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49d65 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.911090:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49d65 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826289.911180:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e45:0x0]: returning size 6 00000004:00200000:1.0:1787826289.911186:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:1.0:1787826289.911233:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f49d5e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801115 expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.911268:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49d65 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801115 expref: 13 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.911280:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7961614000) 00010000:00010000:1.0:1787826289.911284:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826289.911288:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826289.911366:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224052:x1874663332547072:12345-192.168.203.56@tcp:101:rm.0 Request processed in 467us (1129us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.911374:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68290 00000100:00100000:3.0:1787826289.914383:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.915188:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332547200 00000100:00100000:1.0:1787826289.915203:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332547200 00000100:00100000:1.0:1787826289.915305:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11026 00000100:00100000:1.0:1787826289.915318:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332547200:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:1.0:1787826289.915342:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826289.915375:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49d57 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080110e expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.915401:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7952ca9200) 00010000:00010000:1.0:1787826289.915416:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826289.915489:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332547200:12345-192.168.203.56@tcp:103:rm.0 Request processed in 173us (1118us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.915498:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11026 00000100:00100000:1.0:1787826289.919862:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826289.920223:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332547456 00000100:00100000:3.0:1787826289.920238:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332547456 00000100:00100000:3.0:1787826289.920521:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68291 00000100:00100000:3.0:1787826289.920536:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224052:x1874663332547456:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826289.920610:0:8752:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826289.920646:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f49d65 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e45:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801115 expref: 13 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.920668:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7961614000) 00010000:00010000:3.0:1787826289.920683:0:8752:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826289.920751:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.920761:0:8752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.920772:0:8752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826289.920811:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851243100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346144896:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787826289.920891:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.920936:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.922372:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346144896 00000100:00100000:2.0:1787826289.922389:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346144896 00000100:00100000:2.0:1787826289.922470:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23409 00000100:00100000:2.0:1787826289.922482:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8d450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346144896:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787826289.922511:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826289.922534:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49d6c expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.922573:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.922585:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49d6c expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.922630:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.922646:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 9 pid: 35094 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826289.922660:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346145024:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826289.922764:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b1e8900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826289.927574:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346145024:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826289.927627:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49d6c expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.927644:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961614800) 00010000:00010000:3.0:1787826289.927647:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826289.927651:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49d6c expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.927662:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:3.0:1787826289.927737:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8d450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:8752:x1874663346144896:12345-0@lo:101:mdt00_003.0 Request processed in 5257us (6845us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.927746:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23409 00000100:00100000:3.0:1787826289.928091:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851243100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346144896:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787826289.928141:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.928154:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826289.928411:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.928459:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332547584 00000100:00100000:1.0:1787826289.928468:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332547584 00010000:00010000:1.0:1787826289.928514:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 10 pid: 35094 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.928949:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21877 00000100:00100000:1.0:1787826289.929019:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332547584:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826289.929042:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826289.929051:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 10 pid: 35094 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929064:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 10 pid: 35094 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929090:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f49d18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f9 expref: 10 pid: 35094 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929134:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961614800) 00010000:00010000:1.0:1787826289.929140:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49d6c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929150:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49d6c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929159:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49d6c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929197:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49d6c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.929213:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346145152:0@lo:105: 00000100:00100000:1.0:1787826289.929268:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.929316:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78510579c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826289.929357:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826289.929372:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346145152 00000100:00100000:3.0:1787826289.929378:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346145152 00000100:00100000:3.0:1787826289.929420:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7825 00000100:00100000:3.0:1787826289.929427:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c78c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346145152:12345-0@lo:105: 00010000:00010000:3.0:1787826289.929440:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.929452:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.929462:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 5/0,1 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.929470:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.929477:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.929486:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.929513:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69e00) 00010000:00010000:1.0:1787826289.929530:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929540:0:8752:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826289.929557:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c78c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346145152:12345-0@lo:105: Request processed in 131us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.929565:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7825 00000004:00080000:1.0:1787826289.929572:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18e:0x0] 00010000:00010000:1.0:1787826289.929614:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929623:0:8752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.929633:0:8752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826289.929634:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346145152:0@lo:105: 00010000:00010000:3.0:1787826289.929646:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826289.929650:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145280:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787826289.929693:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332547584:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 734us (1284us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826289.929700:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21877 00000100:00100000:1.0:1787826289.929824:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.929882:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826289.929908:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346145280 00000100:00100000:0.0:1787826289.929916:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346145280 00000100:00100000:0.0:1787826289.929965:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23410 00000100:00100000:0.0:1787826289.929975:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:8752:x1874663346145280:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787826289.929998:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826289.930017:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49d81 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49d7a expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.930039:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49d81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49d7a expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.930052:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49d81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49d7a expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.930059:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f794b57a000) 00010000:00010000:0.0:1787826289.930073:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826289.930075:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826289.930152:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346145280:12345-0@lo:101:mdt00_003.0 Request processed in 178us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826289.930160:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23410 00000100:00100000:1.0:1787826289.930213:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145280:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787826289.930238:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49d81 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.930251:0:8752:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d81 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826289.930276:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18e:0x0] 00000100:00100000:1.0:1787826289.930341:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145408:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826289.930393:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.930420:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826289.930541:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346145408 00000100:00100000:0.0:1787826289.930548:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346145408 00000100:00100000:0.0:1787826289.930593:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23411 00000100:00100000:0.0:1787826289.930602:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346145408:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787826289.930691:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346145408:12345-0@lo:1000:mdt00_003.0 Request processed in 91us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826289.930699:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23411 00000100:00100000:0.0:1787826289.930737:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145408:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787826289.930816:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145536:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787826289.930850:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826289.930872:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826289.930879:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346145536 00000100:00100000:1.0:1787826289.930888:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346145536 00000100:00100000:1.0:1787826289.930946:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23412 00000100:00100000:1.0:1787826289.930955:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346145536:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826289.931066:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346145536:12345-0@lo:1000:mdt00_003.0 Request processed in 110us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.931073:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23412 00000100:00100000:2.0:1787826289.933933:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145536:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787826289.934493:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145664:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787826289.934589:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826289.934631:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7952960840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826289.934766:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346145664 00000100:00100000:0.0:1787826289.934776:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346145664 00000100:00100000:0.0:1787826289.934861:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23413 00000100:00100000:0.0:1787826289.934874:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346145664:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787826289.935025:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346145664:12345-0@lo:1000:mdt00_003.0 Request processed in 153us (435us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826289.935036:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23413 00000100:00100000:3.0:1787826289.935203:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145664:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787826289.935331:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.935344:0:8752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.935356:0:8752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826289.935374:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145792:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787826289.935408:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826289.935745:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.935843:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346145792 00000100:00100000:2.0:1787826289.935851:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346145792 00000100:00100000:2.0:1787826289.935904:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23414 00000100:00100000:2.0:1787826289.935913:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346145792:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787826289.935927:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826289.935950:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d8f lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49d88 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.935983:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d8f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49d88 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.935998:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d8f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49d88 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.936008:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:2.0:1787826289.936011:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826289.936016:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826289.936062:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:8752:x1874663346145792:12345-0@lo:101:mdt00_003.0 Request processed in 150us (653us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826289.936082:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23414 00000100:00100000:3.0:1787826289.936334:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346145792:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787826289.936354:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.936367:0:8752:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.936411:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.936421:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826289.936431:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787826289.936907:0:8752:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].220, 64 off22272 00000004:00080000:3.0:1787826289.936926:0:8752:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8102148819750ns, next at 8103148816387ns, rc = 0 00000040:00080000:3.0:1787826289.936969:0:8752:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1183, 384 off1075936 00000040:00080000:3.0:1787826289.937044:0:8752:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1502, 464 off1516368 00000004:00080000:3.0:1787826289.937051:0:8752:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784753b600 with version 6797 00000004:00080000:2.0:1787826289.939224:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6797 rpc_version 6797 00000020:00100000:2.0:1787826289.939372:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826289.939386:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977dcbb80 x1874663346146048/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826289.939420:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346146048:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826289.939518:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826289.939575:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524050c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826289.939600:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346146048 00000100:00100000:1.0:1787826289.939620:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346146048 00000100:00100000:1.0:1787826289.939694:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23415 00000100:00100000:1.0:1787826289.939705:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346146048:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826289.940178:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346146048:12345-0@lo:1000:osp_up0-1.0 Request processed in 472us (657us total) trans 8589957338 rc 0/0 00000100:00100000:1.0:1787826289.940191:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23415 00000100:00100000:1.0:1787826289.940270:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346146048:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826289.940316:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.941807:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.941831:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.941841:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.941849:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.941856:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943414:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826289.943454:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943464:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943477:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943484:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826289.943490:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943498:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943514:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7961d81100) 00010000:00010000:1.0:1787826289.943517:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826289.943522:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943529:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f49d96 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943548:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943557:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.943564:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945712:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826289.945733:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945748:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826289.945753:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945764:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945788:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7961d81100) 00010000:00010000:2.0:1787826289.945792:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826289.945811:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945819:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f49d9d lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e45:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945848:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826289.945853:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945861:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826289.945867:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826289.945877:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945935:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945944:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826289.945976:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.945985:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826289.945987:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826289.945991:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826289.946010:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826289.946036:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845aff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346146176:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826289.948467:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826289.948615:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346146176 00000100:00100000:0.0:1787826289.948627:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346146176 00000100:00100000:0.0:1787826289.948716:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11027 00000100:00100000:0.0:1787826289.948728:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346146176:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826289.948748:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826289.948780:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49d8f lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49d88 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.948811:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e45:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:0.0:1787826289.948825:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826289.948885:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346146176:12345-0@lo:103:ldlm_bl.0 Request processed in 160us (428us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826289.948894:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11027 00000100:00100000:0.0:1787826289.948947:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845aff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346146176:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826289.949315:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826289.949324:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787826289.949341:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f49d88 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e45:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49d8f expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826289.949389:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18e:0x0] 00010000:00010000:1.0:1787826289.949401:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.949410:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.949419:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.949448:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826289.949455:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18e:0x0] 00010000:00010000:1.0:1787826289.949461:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49d81 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.949470:0:8752:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49d81 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.949479:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826289.949490:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.949501:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826289.949509:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826289.949519:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.949582:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.949582:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224052:x1874663332547456:12345-192.168.203.56@tcp:36:rm.0 Request processed in 29040us (29723us total) trans 4294985353 rc 0/0 00010000:00010000:0.0:1787826289.949610:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826289.949612:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68291 00010000:00010000:0.0:1787826289.949641:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 1/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.949653:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69e00/0xe21309aa81f49d6c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49d73 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.949667:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:0.0:1787826289.949678:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826289.949682:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826289.949685:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826289.949689:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346146304:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826289.949758:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826289.953002:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346146304 00000100:00100000:0.0:1787826289.953023:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346146304 00000100:00100000:0.0:1787826289.953151:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11028 00000100:00100000:0.0:1787826289.953165:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd81850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346146304:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826289.953185:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826289.953218:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49d73 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49d6c expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.953240:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f7961614800) 00010000:00010000:0.0:1787826289.953253:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826289.953367:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd81850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346146304:12345-0@lo:103:ldlm_bl.0 Request processed in 206us (3608us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826289.953383:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11028 00000100:00100000:0.0:1787826289.953500:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346146304:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826289.953849:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826289.953895:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.953961:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332547712 00000100:00100000:1.0:1787826289.953970:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332547712 00000100:00100000:1.0:1787826289.954025:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11029 00000100:00100000:1.0:1787826289.954036:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332547712:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:1.0:1787826289.954048:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826289.954079:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49bc8 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801090 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.954097:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:1.0:1787826289.954101:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826289.954175:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332547712:12345-192.168.203.56@tcp:103:rm.0 Request processed in 141us (329us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.954183:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11029 00000100:00100000:0.0:1787826289.954717:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332547968 00000100:00100000:0.0:1787826289.954726:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332547968 00000100:00100000:0.0:1787826289.954780:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9316 00000100:00100000:0.0:1787826289.954791:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224052:x1874663332547968:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:0.0:1787826289.955169:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:224052:x1874663332547968:12345-192.168.203.56@tcp:35:rm.0 Request processed in 377us (1270us total) trans 4294985354 rc 0/0 00000100:00100000:0.0:1787826289.955178:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9316 00000100:00100000:2.0:1787826289.958908:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826289.959286:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332548224 00000100:00100000:1.0:1787826289.959300:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332548224 00000100:00100000:1.0:1787826289.959521:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68292 00000100:00100000:1.0:1787826289.959535:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:224052:x1874663332548224:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826289.959619:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826289.959653:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f49d34 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e44:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801107 expref: 14 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.959676:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f49cfc lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010eb expref: 13 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.959691:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f49bd6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801097 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.959703:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7961614600) 00010000:00010000:1.0:1787826289.959709:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826289.959771:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.959782:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.959794:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.959836:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346146432:0@lo:101:mdt00_011.0 00000100:00100000:1.0:1787826289.959910:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.960001:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826289.960470:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346146432 00000100:00100000:0.0:1787826289.960481:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346146432 00000100:00100000:1.0:1787826289.960641:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23416 00000100:00100000:1.0:1787826289.960654:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346146432:12345-0@lo:101:mdt00_011.0 00010000:00010000:1.0:1787826289.960676:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826289.960698:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49da4 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.960733:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.960741:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.960750:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.960781:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.960812:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 7 pid: 35092 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.960826:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346146560:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826289.960930:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.960940:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 7 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.960949:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346146688:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826289.960974:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.964784:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826289.964854:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332548352 00000100:00100000:2.0:1787826289.964864:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332548352 00010000:00010000:2.0:1787826289.964919:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 8 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826289.964964:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21878 00000100:00100000:2.0:1787826289.964977:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332548352:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826289.964996:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826289.965004:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 8 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.965017:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 8 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.965047:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826289.965055:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.965078:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826289.965164:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332548352:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 188us (394us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826289.965173:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21878 00000100:00100000:1.0:1787826289.965415:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346146560:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826289.965433:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f49bac lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801089 expref: 7 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.965448:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346146688:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826289.965483:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.965492:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826289.965495:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826289.965498:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.965510:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826289.965568:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346146432:12345-0@lo:101:mdt00_011.0 Request processed in 4917us (5659us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826289.965576:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23416 00000100:00100000:2.0:1787826289.965762:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346146432:0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826289.965782:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.965794:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826289.970795:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826289.970861:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332548480 00000100:00100000:2.0:1787826289.970875:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332548480 00010000:00010000:2.0:1787826289.970930:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 8 pid: 35092 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826289.970982:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21879 00000100:00100000:2.0:1787826289.970994:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332548480:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826289.971013:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826289.971022:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 8 pid: 35092 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.971040:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 8 pid: 35092 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.971067:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49c15 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010b3 expref: 8 pid: 35092 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.976561:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826289.976584:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.976606:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.976618:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826289.976830:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826289.976850:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346146816:0@lo:105: 00000100:00100000:1.0:1787826289.976936:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826289.977048:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.981707:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346146816 00000100:00100000:2.0:1787826289.981731:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346146816 00000100:00100000:2.0:1787826289.981815:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7826 00000100:00100000:2.0:1787826289.981828:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346146816:12345-0@lo:105: 00010000:00010000:2.0:1787826289.981850:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.981865:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.981874:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.981885:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.981894:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.981904:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.981931:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf000) 00000100:00100000:2.0:1787826289.982034:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346146816:12345-0@lo:105: Request processed in 208us (5097us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826289.982043:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7826 00000100:00100000:2.0:1787826289.983602:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346146816:0@lo:105: 00010000:00010000:2.0:1787826289.983637:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826289.983710:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332548480:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 12715us (12930us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826289.983721:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21879 00010000:00010000:2.0:1787826289.985522:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.985546:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826289.985585:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826289.985656:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.985665:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.985680:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826289.985709:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98c380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346146944:0@lo:101:mdt00_011.0 00000100:00100000:2.0:1787826289.985808:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826289.985850:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.985881:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346146944 00000100:00100000:2.0:1787826289.985890:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346146944 00000100:00100000:2.0:1787826289.985950:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23417 00000100:00100000:2.0:1787826289.985961:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346146944:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826289.985984:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826289.986005:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49db9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49db2 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.986034:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49db9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49db2 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.986051:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49db9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49db2 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.986061:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9a00) 00010000:00010000:2.0:1787826289.986064:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826289.986069:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826289.986135:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346146944:12345-0@lo:101:mdt00_011.0 Request processed in 174us (337us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826289.986142:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23417 00000100:00100000:2.0:1787826289.986191:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98c380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346146944:0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826289.986207:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826289.986218:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826289.986234:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826289.986274:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98c380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147072:0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826289.986303:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826289.987464:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346147072 00000100:00100000:0.0:1787826289.987479:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346147072 00000100:00100000:0.0:1787826289.987543:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23418 00000100:00100000:0.0:1787826289.987554:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346147072:12345-0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826289.987676:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346147072:12345-0@lo:1000:mdt00_011.0 Request processed in 123us (1371us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826289.987686:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23418 00000100:00100000:2.0:1787826289.988950:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524056c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.988997:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98c380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147072:0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826289.989134:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98c380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147200:0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826289.989205:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826289.989230:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524056c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.989261:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346147200 00000100:00100000:2.0:1787826289.989268:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346147200 00000100:00100000:2.0:1787826289.989314:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23419 00000100:00100000:2.0:1787826289.990890:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346147200:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826289.991097:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346147200:12345-0@lo:1000:mdt00_011.0 Request processed in 222us (1892us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826289.995262:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98c380 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147200:0@lo:1000:mdt00_011.0 00010000:00010000:0.0:1787826289.995402:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.995415:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.995427:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826289.995450:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147328:0@lo:101:mdt00_011.0 00000100:00100000:0.0:1787826289.995530:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826289.995567:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826289.995594:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346147328 00000100:00100000:0.0:1787826289.995603:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346147328 00000100:00100000:0.0:1787826289.995662:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23420 00000100:00100000:0.0:1787826289.995673:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346147328:12345-0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826289.995695:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826289.995714:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f49dc7 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49dc0 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.995750:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.995762:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f49dc7 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49dc0 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.995791:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826289.995820:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 6 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826289.995833:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346147456:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826289.995914:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826289.996176:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23419 00000100:00100000:0.0:1787826290.001896:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826290.001938:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332548608 00000100:00100000:0.0:1787826290.001950:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332548608 00010000:00010000:0.0:1787826290.002015:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 5/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 7 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.002063:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21880 00000100:00100000:0.0:1787826290.002087:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332548608:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826290.002127:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826290.002136:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 5/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 7 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.002151:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 7 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.002179:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:0.0:1787826290.002186:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f49dc7 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49dc0 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.002198:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f49dc7 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49dc0 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.002208:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f49dc7 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f49dc0 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.002249:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f49dc7 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49dc0 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.002264:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346147584:0@lo:105: 00000100:00100000:0.0:1787826290.002321:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.002364:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ff00 going to sleep for 11 seconds 00000100:00080000:0.0:1787826290.002877:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787826290.002952:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f49c31 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010ba expref: 7 pid: 35095 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.002971:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346147456:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826290.003004:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f49dc7 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49dc0 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.003015:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:0.0:1787826290.003018:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826290.003021:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826290.003165:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346147328:12345-0@lo:101:mdt00_011.0 Request processed in 7491us (7632us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.003175:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23420 00000100:00100000:3.0:1787826290.003209:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346147584 00000100:00100000:3.0:1787826290.003218:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346147584 00000100:00100000:0.0:1787826290.003238:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147328:0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826290.003260:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826290.003268:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7827 00010000:00010000:0.0:1787826290.003274:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826290.003284:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346147584:12345-0@lo:105: 00010000:00010000:3.0:1787826290.003298:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.003308:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.003318:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.003326:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.003334:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.003344:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.003368:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43e00) 00000100:00100000:3.0:1787826290.003417:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346147584:12345-0@lo:105: Request processed in 140us (1095us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.003425:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7827 00000100:00100000:3.0:1787826290.003500:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346147584:0@lo:105: 00010000:00010000:3.0:1787826290.003515:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826290.004564:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332548608:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1485us (1684us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.004750:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21880 00010000:00010000:0.0:1787826290.005670:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.005691:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.005751:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.005768:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.005783:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.007197:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.007227:0:35094:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.007236:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.007291:0:35094:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.007309:0:35094:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 11 pid: 35093 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.007587:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001_UUID:35094:1874663346147712:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826290.007750:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.017248:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826290.017292:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001_UUID:35094:1874663346147712:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826290.017340:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.017547:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332548736 00000100:00100000:2.0:1787826290.017556:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332548736 00010000:00010000:2.0:1787826290.017606:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 12 pid: 35093 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.017654:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21881 00000100:00100000:2.0:1787826290.017665:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332548736:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826290.017685:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826290.017693:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 12 pid: 35093 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.017708:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 12 pid: 35093 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.017734:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f49d0a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608010f2 expref: 12 pid: 35093 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.017760:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:2.0:1787826290.017766:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.017777:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.017786:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.017831:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826290.017897:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332548736:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 233us (666us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.017905:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21881 00010000:00010000:3.0:1787826290.018277:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.018288:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.018347:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.018356:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.018368:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826290.018392:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242a00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147840:0@lo:101:mdt00_011.0 00000100:00100000:3.0:1787826290.018461:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.018625:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346147840 00000100:00100000:0.0:1787826290.018634:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346147840 00000100:00100000:2.0:1787826290.018673:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.018693:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23421 00000100:00100000:0.0:1787826290.018704:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346147840:12345-0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826290.018725:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826290.018743:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49de3 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49ddc expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.018770:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49de3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49ddc expref: 14 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.018803:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49de3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49ddc expref: 14 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.018813:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f794b57aa00) 00010000:00010000:0.0:1787826290.018816:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826290.018820:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826290.019021:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346147840:12345-0@lo:101:mdt00_011.0 Request processed in 319us (560us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.019029:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23421 00000100:00100000:2.0:1787826290.019363:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242a00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147840:0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826290.019395:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.019409:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.020648:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147968:0@lo:403:mdt00_011.0 00000100:00100000:2.0:1787826290.020842:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826290.021549:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.024389:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346147968 00000100:00100000:0.0:1787826290.024414:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346147968 00000100:00100000:1.0:1787826290.025806:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23422 00000100:00100000:1.0:1787826290.025822:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346147968:12345-0@lo:403:mdt00_011.0 00000100:00100000:1.0:1787826290.028250:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346147968:12345-0@lo:403:mdt00_011.0 Request processed in 2419us (7395us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826290.028269:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23422 00000100:00100000:2.0:1787826290.028487:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346147968:0@lo:403:mdt00_011.0 00000040:00080000:2.0:1787826290.029871:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1184, 952 off1076888 00000040:00080000:2.0:1787826290.030143:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1503, 1032 off1517400 00000004:00080000:2.0:1787826290.030151:0:35094:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852405d80 with version 6798 00000004:00080000:1.0:1787826290.030743:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6798 rpc_version 6798 00000020:00100000:1.0:1787826290.030804:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826290.030819:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f79785e5880 x1874663346148224/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2608/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826290.030850:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346148224:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826290.030936:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826290.031028:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.031282:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346148224 00000100:00100000:0.0:1787826290.031293:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346148224 00000100:00100000:1.0:1787826290.031498:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23423 00000100:00100000:1.0:1787826290.031511:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346148224:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826290.032348:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346148224:12345-0@lo:1000:osp_up0-1.0 Request processed in 834us (1406us total) trans 8589957339 rc 0/0 00000100:00100000:1.0:1787826290.032361:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23423 00000100:00100000:2.0:1787826290.032572:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346148224:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787826290.032644:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e44:0x0]: returning size 4096 00010000:00010000:1.0:1787826290.032673:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032685:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032693:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032701:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826290.032706:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032716:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032735:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x190:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:1.0:1787826290.032739:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826290.032744:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032752:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49dd5 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032782:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032792:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032813:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032839:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826290.032851:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18e:0x0] 00010000:00010000:1.0:1787826290.032861:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032868:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032874:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032880:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826290.032883:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032889:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032898:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7961614600) 00010000:00010000:1.0:1787826290.032901:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826290.032905:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032911:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f49dce lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032922:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032930:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032937:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.032993:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787826290.033007:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826290.033012:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.033020:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.033027:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033097:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1787826290.033122:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033131:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826290.033136:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.033146:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033188:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033195:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826290.033233:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 1/0,0 mode: --/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033240:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f49ddc lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49de3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033258:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826290.033261:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033271:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826290.033275:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.033282:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033299:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033305:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826290.033310:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826290.033346:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346148352:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826290.033414:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826290.033468:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826290.033475:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b663b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346148480:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826290.033486:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f49dc0 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49dc7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033506:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1787826290.033506:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826290.033512:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033520:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826290.033526:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.033533:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826290.033550:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346148352 00010000:00010000:2.0:1787826290.033559:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826290.033560:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346148352 00010000:00010000:2.0:1787826290.033569:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826290.033591:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.033600:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826290.033602:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826290.033605:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826290.033616:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346148480 00000100:00100000:3.0:1787826290.033621:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346148480 00000100:00100000:2.0:1787826290.033624:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346148608:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826290.033646:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11030 00000100:00100000:3.0:1787826290.033658:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3654:x1874663346148352:12345-0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826290.033666:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826290.033675:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826290.033699:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f49de3 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49ddc expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.033717:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x0 (ffff9f794b57aa00) 00000100:00100000:2.0:1787826290.033718:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346148608 00000100:00100000:2.0:1787826290.033724:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346148608 00010000:00010000:3.0:1787826290.033726:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826290.033763:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3654:x1874663346148352:12345-0@lo:103:ldlm_bl.0 Request processed in 107us (349us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.033765:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11031 00000100:00100000:3.0:1787826290.033775:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11030 00000100:00100000:2.0:1787826290.033780:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd83450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+16:3654:x1874663346148480:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826290.033790:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826290.033815:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11032 00010000:00010000:2.0:1787826290.033818:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f49dc7 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e44:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49dc0 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826290.033825:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346148608:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826290.033837:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:3.0:1787826290.033845:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826290.033848:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826290.033861:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49dab lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49da4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.033876:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826290.033880:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826290.033884:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd83450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346148480:12345-0@lo:103:ldlm_bl.0 Request processed in 106us (377us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.033893:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11031 00000100:00100000:3.0:1787826290.033914:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346148608:12345-0@lo:103:ldlm_bl.0 Request processed in 91us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.033922:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11032 00000100:00100000:3.0:1787826290.033974:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b663b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346148480:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826290.033986:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346148608:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826290.033990:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346148352:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826290.034128:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787826290.034137:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f49da4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49dab expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826290.034159:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826290.034165:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.034172:0:35094:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826290.034261:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224052:x1874663332548224:12345-192.168.203.56@tcp:36:rm.0 Request processed in 74726us (75467us total) trans 4294985355 rc 0/0 00000100:00100000:1.0:1787826290.034269:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68292 00010000:00010000:2.0:1787826290.034676:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826290.039503:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826290.101855:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826290.101932:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332548864 00000100:00100000:2.0:1787826290.101946:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332548864 00000100:00100000:2.0:1787826290.102021:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68293 00000100:00100000:2.0:1787826290.102034:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224060:x1874663332548864:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826290.102086:0:11629:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345618944 00000100:00100000:2.0:1787826290.102206:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224060:x1874663332548864:12345-192.168.203.56@tcp:41:stat.0 Request processed in 175us (366us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.102215:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68293 00000100:00100000:0.0:1787826290.156868:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346148736:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826290.156993:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826290.157235:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346148736 00000100:00100000:1.0:1787826290.157247:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346148736 00000100:00100000:1.0:1787826290.157357:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6207 00000100:00100000:1.0:1787826290.157370:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346148736:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826290.157439:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826290.157527:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332548992 00000100:00100000:2.0:1787826290.157533:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332548992 00002000:00000024:1.0:1787826290.157575:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826290.157581:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68294 00000100:00100000:2.0:1787826290.157591:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224064:x1874663332548992:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826290.157608:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826290.157641:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49dea lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080111c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826290.157647:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346148736:12345-0@lo:13:osp-pre-1-1.0 Request processed in 279us (653us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826290.157662:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6207 00000100:00100000:3.0:1787826290.157814:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346148736:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1787826290.157978:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.157992:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.158002:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826290.158065:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826290.158078:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826290.158147:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.158179:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49dea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.158201:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.158211:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826290.158213:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826290.158216:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826290.158264:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224064:x1874663332548992:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 675us (827us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.158271:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68294 00000100:00100000:3.0:1787826290.161943:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826290.162146:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332549120 00000100:00100000:0.0:1787826290.162158:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332549120 00000100:00100000:0.0:1787826290.162236:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68295 00000100:00100000:0.0:1787826290.162249:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224064:x1874663332549120:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826290.162275:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826290.162302:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49df8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801123 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.162397:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.162415:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.162429:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.162441:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.162452:0:35095:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49db9 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49db2 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.162461:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.162503:0:35095:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49db9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49db2 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.162522:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301c00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346148864:0@lo:104: 00000100:00100000:0.0:1787826290.162589:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.162638:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826290.162820:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346148864 00000100:00100000:2.0:1787826290.162826:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346148864 00000100:00100000:2.0:1787826290.162874:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7828 00000100:00100000:2.0:1787826290.162883:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346148864:12345-0@lo:104: 00010000:00010000:2.0:1787826290.162897:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.162979:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826290.162985:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346148864:12345-0@lo:104: Request processed in 104us (396us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.162993:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7828 00010000:00010000:2.0:1787826290.163021:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826290.163028:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.163042:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826290.163049:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.163059:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.163134:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.163143:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826290.163166:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.163173:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f49db2 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49db9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.163195:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826290.163197:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826290.163199:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826290.163208:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826290.163220:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346148992:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826290.163256:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826290.163298:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346148992 00000100:00100000:2.0:1787826290.163303:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346148992 00010000:00010000:2.0:1787826290.163324:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49db9 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49db2 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.163352:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21882 00000100:00100000:2.0:1787826290.163359:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346148992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826290.163370:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826290.163450:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301c00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346148864:0@lo:104: 00010000:00010000:1.0:1787826290.163482:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826290.167697:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957339 is committed 00010000:00010000:2.0:1787826290.169446:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49db9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49db2 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.169477:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9a00) 00010000:00010000:2.0:1787826290.169484:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.169495:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.169504:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.169790:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826290.170218:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346148992:12345-0@lo:103:ldlm_bl.0 Request processed in 6854us (6956us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.170232:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21882 00000100:00100000:3.0:1787826290.170653:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346148992:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826290.170746:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.170760:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.170934:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.170943:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.170949:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.170956:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.170962:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826290.170966:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.170974:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.170988:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826290.170991:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826290.170995:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.171001:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f49dff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.171015:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.171021:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.171026:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.171032:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826290.171035:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.171041:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.171048:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9a00) 00010000:00010000:0.0:1787826290.171050:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826290.171053:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.171059:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f49e06 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.173512:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49df8 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801123 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.173537:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826290.173540:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826290.173545:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49df8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801123 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.173563:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826290.173670:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224064:x1874663332549120:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 11422us (11739us total) trans 0 rc 301/301 00000100:00100000:0.0:1787826290.173688:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68295 00000100:00100000:0.0:1787826290.180426:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826290.180736:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332549248 00000100:00100000:3.0:1787826290.180749:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332549248 00000100:00100000:3.0:1787826290.180832:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68296 00000100:00100000:3.0:1787826290.181207:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224064:x1874663332549248:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826290.181242:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826290.181280:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49e0d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080112a expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.181377:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.181394:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.181407:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826290.181470:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826290.181475:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826290.181499:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.181537:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49e0d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.181564:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.181573:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826290.181576:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826290.181581:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826290.181830:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224064:x1874663332549248:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 629us (1418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.181839:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68296 00000100:00100000:3.0:1787826290.184945:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826290.185090:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332549504 00000100:00100000:1.0:1787826290.185102:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332549504 00000100:00100000:1.0:1787826290.185230:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68297 00000100:00100000:1.0:1787826290.185243:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224064:x1874663332549504:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:1.0:1787826290.185425:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346149120:0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826290.185491:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826290.185537:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826290.185557:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346149120 00000100:00100000:3.0:1787826290.185562:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346149120 00000100:00100000:3.0:1787826290.185605:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23424 00000100:00100000:3.0:1787826290.185614:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346149120:12345-0@lo:1000:mdt00_006.0 00000100:00080000:0.0:1787826290.185623:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826290.185714:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346149120:12345-0@lo:1000:mdt00_006.0 Request processed in 101us (221us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.185731:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23424 00000100:00100000:0.0:1787826290.185820:0:24702:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957339 gen 2 00000004:00080000:0.0:1787826290.185827:0:24702:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957337 committed transno 8589957339 00000004:00080000:0.0:1787826290.185857:0:24702:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957338 committed transno 8589957339 00000004:00080000:0.0:1787826290.185873:0:24702:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957339 committed transno 8589957339 00000100:00100000:0.0:1787826290.185900:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346149120:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.185955:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346149248:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.185987:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.186016:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826290.186033:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346149248 00000100:00100000:1.0:1787826290.186038:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346149248 00000100:00100000:1.0:1787826290.186192:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23425 00000100:00100000:1.0:1787826290.186202:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346149248:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826290.186389:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346149248:12345-0@lo:1000:mdt00_006.0 Request processed in 188us (399us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826290.186397:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23425 00000100:00100000:2.0:1787826290.186482:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346149248:0@lo:1000:mdt00_006.0 00010000:00010000:2.0:1787826290.186550:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.186562:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.186572:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.186587:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346149376:0@lo:101:mdt00_006.0 00000100:00100000:2.0:1787826290.186616:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.186943:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346149376 00000100:00100000:2.0:1787826290.186949:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.186950:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346149376 00000100:00100000:0.0:1787826290.187001:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23426 00000100:00100000:0.0:1787826290.187010:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24702:x1874663346149376:12345-0@lo:101:mdt00_006.0 00010000:00010000:0.0:1787826290.187033:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826290.187053:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49e1b expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.187085:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.187095:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.187549:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.187589:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.187609:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 7 pid: 24703 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.187623:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346149504:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826290.187762:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.187773:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 6 pid: 35098 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.187868:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346149632:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826290.187896:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826290.195703:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826290.196595:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346149504:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826290.196617:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346149632:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826290.196674:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.196691:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826290.196695:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826290.196699:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.196712:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826290.196834:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:24702:x1874663346149376:12345-0@lo:101:mdt00_006.0 Request processed in 9822us (10215us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.196845:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23426 00000100:00100000:3.0:1787826290.197322:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826290.197564:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332549632 00000100:00100000:0.0:1787826290.197575:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332549632 00010000:00010000:0.0:1787826290.197620:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 8 pid: 24703 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.197662:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332549760 00000100:00100000:0.0:1787826290.197667:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332549760 00010000:00010000:0.0:1787826290.197682:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 7 pid: 35098 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.197703:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21883 00000100:00100000:0.0:1787826290.197713:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332549632:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826290.197732:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826290.197740:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 8 pid: 24703 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197753:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 8 pid: 24703 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197790:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f49e14 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080112a expref: 8 pid: 24703 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197818:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826290.197824:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197838:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826290.197890:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332549632:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 178us (2204us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.197898:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21883 00000100:00100000:0.0:1787826290.197918:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21884 00000100:00100000:0.0:1787826290.197925:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332549760:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826290.197935:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826290.197940:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 7 pid: 35098 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197950:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 7 pid: 35098 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197965:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49df1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080111c expref: 7 pid: 35098 timeout: 8202 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197975:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826290.197979:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197988:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.197997:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.198039:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.198054:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346149760:0@lo:105: 00000100:00100000:1.0:1787826290.198245:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346149376:0@lo:101:mdt00_006.0 00010000:00010000:1.0:1787826290.198282:0:24702:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.198293:0:24702:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.198536:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.198577:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826290.199457:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346149760 00000100:00100000:3.0:1787826290.199469:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346149760 00000100:00100000:3.0:1787826290.199533:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7829 00000100:00100000:3.0:1787826290.199544:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346149760:12345-0@lo:105: 00010000:00010000:3.0:1787826290.199560:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.199573:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.199582:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.199592:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.199600:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.199609:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.199642:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef600) 00000100:00100000:3.0:1787826290.199710:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346149760:12345-0@lo:105: Request processed in 168us (1174us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.199719:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7829 00000100:00100000:0.0:1787826290.200283:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346149760:0@lo:105: 00010000:00010000:0.0:1787826290.200459:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826290.200528:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332549760:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2600us (3216us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.200538:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21884 00010000:00010000:0.0:1787826290.200580:0:24702:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.200592:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826290.200619:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826290.200676:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.200746:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.200760:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.200793:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346149888:0@lo:101:mdt00_006.0 00000100:00100000:0.0:1787826290.200938:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.201297:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826290.205657:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346149888 00000100:00100000:2.0:1787826290.205679:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346149888 00000100:00100000:2.0:1787826290.205945:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23427 00000100:00100000:2.0:1787826290.205960:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:24702:x1874663346149888:12345-0@lo:101:mdt00_006.0 00010000:00010000:2.0:1787826290.205989:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826290.206022:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49e30 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49e29 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.206074:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49e30 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49e29 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.206092:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49e30 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e29 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.206102:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9e00) 00010000:00010000:2.0:1787826290.206131:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826290.206137:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826290.206236:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346149888:12345-0@lo:101:mdt00_006.0 Request processed in 279us (5297us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.206243:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23427 00000100:00100000:0.0:1787826290.206422:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346149888:0@lo:101:mdt00_006.0 00010000:00010000:0.0:1787826290.206448:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.206462:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826290.206495:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826290.206560:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150016:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.206613:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.207596:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826290.207697:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346150016 00000100:00100000:3.0:1787826290.207707:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346150016 00000100:00100000:3.0:1787826290.207885:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23428 00000100:00100000:3.0:1787826290.207898:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150016:12345-0@lo:1000:mdt00_006.0 00000100:00100000:3.0:1787826290.208021:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150016:12345-0@lo:1000:mdt00_006.0 Request processed in 126us (1408us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.208485:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150016:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.208573:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150144:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.208618:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.208692:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826290.208761:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23428 00000100:00100000:3.0:1787826290.208928:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346150144 00000100:00100000:3.0:1787826290.208935:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346150144 00000100:00100000:2.0:1787826290.208986:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23429 00000100:00100000:2.0:1787826290.208999:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150144:12345-0@lo:1000:mdt00_006.0 00000100:00100000:2.0:1787826290.209169:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150144:12345-0@lo:1000:mdt00_006.0 Request processed in 173us (550us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826290.209178:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23429 00000100:00100000:0.0:1787826290.209423:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150144:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.209548:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150272:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.209582:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.209605:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826290.209763:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346150272 00000100:00100000:1.0:1787826290.209770:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346150272 00000100:00100000:1.0:1787826290.209835:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23430 00000100:00100000:1.0:1787826290.209845:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150272:12345-0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826290.209935:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150272:12345-0@lo:1000:mdt00_006.0 Request processed in 92us (352us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826290.209943:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23430 00000100:00100000:0.0:1787826290.210091:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150272:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210181:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150400:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210208:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.210222:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.210237:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346150400 00000100:00100000:0.0:1787826290.210241:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346150400 00000100:00100000:0.0:1787826290.210400:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23431 00000100:00100000:0.0:1787826290.210409:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150400:12345-0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210467:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150400:12345-0@lo:1000:mdt00_006.0 Request processed in 60us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.210474:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23431 00000100:00100000:0.0:1787826290.210510:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150400:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210560:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150528:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210584:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.210599:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.210611:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346150528 00000100:00100000:0.0:1787826290.210615:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346150528 00000100:00100000:0.0:1787826290.210634:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23432 00000100:00100000:0.0:1787826290.210640:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150528:12345-0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210705:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150528:12345-0@lo:1000:mdt00_006.0 Request processed in 65us (120us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.210712:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23432 00000100:00100000:0.0:1787826290.210740:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150528:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210862:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150656:0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210889:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.210905:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.210918:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346150656 00000100:00100000:0.0:1787826290.210921:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346150656 00000100:00100000:0.0:1787826290.210940:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23433 00000100:00100000:0.0:1787826290.210947:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150656:12345-0@lo:1000:mdt00_006.0 00000100:00100000:0.0:1787826290.210996:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24702:x1874663346150656:12345-0@lo:1000:mdt00_006.0 Request processed in 49us (105us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.211003:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23433 00000100:00100000:0.0:1787826290.211032:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346150656:0@lo:1000:mdt00_006.0 00000040:00080000:0.0:1787826290.213168:0:24702:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1185, 2168 off1079056 00000040:00080000:0.0:1787826290.213291:0:24702:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1504, 2368 off1519768 00000004:00080000:0.0:1787826290.213298:0:24702:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784549f300 with version 6799 00000004:00080000:2.0:1787826290.213447:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6799 rpc_version 6799 00000020:00100000:2.0:1787826290.213504:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826290.213519:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b662a00 x1874663346150912/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826290.213551:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b662a00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346150912:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826290.213653:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826290.214976:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346150912 00000100:00100000:1.0:1787826290.214992:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346150912 00000100:00100000:1.0:1787826290.215151:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23434 00000100:00100000:1.0:1787826290.215216:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346150912:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826290.217921:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524050c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826290.218472:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346150912:12345-0@lo:1000:osp_up0-1.0 Request processed in 3306us (4813us total) trans 8589957340 rc 0/0 00000100:00100000:1.0:1787826290.218486:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23434 00000100:00100000:0.0:1787826290.219280:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b662a00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346150912:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826290.219527:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.219548:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.219561:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.219587:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.219594:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.219601:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.219644:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.219652:0:24702:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.219667:0:24702:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.219689:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346151040:0@lo:101:mdt00_006.0 00000100:00100000:2.0:1787826290.219755:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826290.219917:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524050c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826290.219974:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346151040 00000100:00100000:1.0:1787826290.219983:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346151040 00000100:00100000:1.0:1787826290.220491:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23435 00000100:00100000:1.0:1787826290.220506:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:24702:x1874663346151040:12345-0@lo:101:mdt00_006.0 00010000:00010000:1.0:1787826290.220535:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826290.220558:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49e4c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49e45 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.220588:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49e4c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49e45 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.220602:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49e4c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e45 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.220612:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:1.0:1787826290.220616:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826290.220621:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826290.220695:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:24702:x1874663346151040:12345-0@lo:101:mdt00_006.0 Request processed in 194us (939us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826290.220702:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23435 00000100:00100000:2.0:1787826290.220840:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-mdtlov_UUID:24702:1874663346151040:0@lo:101:mdt00_006.0 00010000:00010000:2.0:1787826290.220860:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.220872:0:24702:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.220899:0:24702:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ddaf5b63 reply state 0000000050c2aac3 transno 4294985356 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.220915:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.220920:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:2.0:1787826290.220923:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826290.220926:0:24702:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826290.220933:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.220941:0:24702:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826290.220960:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18f:0x0] 00000004:00010000:2.0:1787826290.220971:0:24702:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ddaf5b63 reply state 0000000050c2aac3 transno 4294985356 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.220979:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.220985:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:2.0:1787826290.220987:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826290.220989:0:24702:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787826290.220995:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e46:0x0]: returning size 4096 00000004:00080000:2.0:1787826290.221006:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826290.221022:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.221029:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.221037:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.221071:0:24702:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826290.221076:0:24702:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826290.221081:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.221089:0:24702:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.221240:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224064:x1874663332549504:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 35997us (36360us total) trans 4294985356 rc 0/0 00000100:00100000:2.0:1787826290.221248:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68297 00010000:00010000:0.0:1787826290.221400:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826290.221409:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.221418:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826290.221425:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826290.221435:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.221475:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.221482:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826290.221515:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.221523:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f49e1b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49e22 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.221541:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826290.221543:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826290.221545:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826290.221554:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826290.221570:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346151168:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826290.221624:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826290.223261:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346151168 00000100:00100000:3.0:1787826290.223280:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346151168 00000100:00100000:3.0:1787826290.223557:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11033 00000100:00100000:3.0:1787826290.223572:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346151168:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826290.223592:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826290.223628:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f49e22 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49e1b expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.223651:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826290.223657:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826290.223746:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346151168:12345-0@lo:103:ldlm_bl.0 Request processed in 178us (2121us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.223754:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11033 00000100:00100000:0.0:1787826290.223880:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346151168:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826290.236996:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826290.240350:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332549888 00000100:00100000:1.0:1787826290.240374:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332549888 00000100:00100000:1.0:1787826290.240488:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68298 00000100:00100000:1.0:1787826290.240501:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224053:x1874663332549888:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826290.240528:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826290.240555:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f49e53 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801131 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.240642:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f49e5a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.240661:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f49e5a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.240674:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f49e5a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826290.240732:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826290.240736:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826290.240761:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f49e5a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.240791:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f49e53 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801131 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.240836:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f49e5a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801131 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.240845:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826290.240848:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826290.240853:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:1.0:1787826290.240936:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224053:x1874663332549888:12345-192.168.203.56@tcp:101:bash.0 Request processed in 437us (3956us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826290.240944:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68298 00000100:00100000:2.0:1787826290.247490:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826290.247641:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332550016 00000100:00100000:1.0:1787826290.247655:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332550016 00000100:00100000:1.0:1787826290.247738:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68299 00000100:00100000:1.0:1787826290.247752:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224053:x1874663332550016:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826290.247780:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826290.247818:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f49e61 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801138 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.247918:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.247934:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.247947:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.247959:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.247972:0:6527:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49e30 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e29 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.247982:0:6527:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.248083:0:6527:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49e30 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49e29 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826290.248100:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346151296:0@lo:104: 00000100:00100000:1.0:1787826290.248204:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826290.248248:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.248328:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346151296 00000100:00100000:0.0:1787826290.248334:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346151296 00000100:00100000:0.0:1787826290.248381:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7830 00000100:00100000:0.0:1787826290.248389:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346151296:12345-0@lo:104: 00010000:00010000:0.0:1787826290.248403:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.248549:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826290.248557:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346151296:12345-0@lo:104: Request processed in 169us (352us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.248564:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7830 00010000:00010000:2.0:1787826290.248607:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826290.248616:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.248625:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826290.248634:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.248644:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.248676:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.248683:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826290.248731:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b663b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346151424:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826290.248767:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.248832:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346151424 00000100:00100000:0.0:1787826290.248836:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346151424 00010000:00010000:0.0:1787826290.248860:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49e30 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49e29 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826290.248869:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346151296:0@lo:104: 00000100:00100000:0.0:1787826290.248886:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21885 00010000:00010000:1.0:1787826290.248894:0:6527:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.248904:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346151424:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826290.248918:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826290.248950:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.248959:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f49e29 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49e30 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.248984:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826290.248987:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826290.248989:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1787826290.253270:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957340 is committed 00010000:00010000:0.0:1787826290.253675:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49e30 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49e29 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.253702:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9e00) 00010000:00010000:0.0:1787826290.253708:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.253721:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.253730:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.253829:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826290.253959:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3652:x1874663346151424:12345-0@lo:103:ldlm_bl.0 Request processed in 5064us (5188us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.253972:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21885 00000100:00100000:3.0:1787826290.254370:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b663b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346151424:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826290.254615:0:6527:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.254627:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826290.254825:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6527:1874663346151552:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826290.254891:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826290.254963:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826290.255402:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346151552 00000100:00100000:0.0:1787826290.255412:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346151552 00000100:00100000:0.0:1787826290.255576:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23436 00000100:00100000:0.0:1787826290.255589:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6527:x1874663346151552:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826290.255709:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6527:x1874663346151552:12345-0@lo:1000:mdt00_000.0 Request processed in 123us (818us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.255717:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23436 00000100:00100000:2.0:1787826290.255789:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6527:1874663346151552:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787826290.255871:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49e76 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.255888:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49e76 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.255899:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49e76 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.255927:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49e76 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.255945:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.255952:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.255959:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.255966:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.255972:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826290.255977:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.255985:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256000:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826290.256003:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826290.256007:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256014:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49e68 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256041:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256047:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256052:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256057:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826290.256060:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256075:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256084:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9e00) 00010000:00010000:2.0:1787826290.256086:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826290.256089:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256094:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49e6f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256165:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f49e61 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801138 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256181:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49e76 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801138 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.256189:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7952ca9f00) 00010000:00010000:2.0:1787826290.256191:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826290.256194:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826290.256261:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224053:x1874663332550016:12345-192.168.203.56@tcp:101:bash.0 Request processed in 8513us (8788us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.256269:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68299 00000100:00100000:3.0:1787826290.261817:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826290.262074:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332550144 00000100:00100000:0.0:1787826290.262088:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332550144 00000100:00100000:0.0:1787826290.262351:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68300 00000100:00100000:0.0:1787826290.262365:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224053:x1874663332550144:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826290.262396:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826290.262425:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f49e7d lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080113f expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.262665:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f49e84 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.262683:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f49e84 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.262696:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f49e84 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826290.262837:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e46:0x0]: returning size 4096 00000004:00200000:0.0:1787826290.262843:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826290.262891:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f49e7d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080113f expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.262913:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f49e84 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080113f expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.262923:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:0.0:1787826290.262926:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826290.262931:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826290.263383:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224053:x1874663332550144:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1019us (1586us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.263392:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68300 00000100:00100000:3.0:1787826290.268711:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826290.269379:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332550272 00000100:00100000:2.0:1787826290.269393:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332550272 00000100:00100000:3.0:1787826290.269575:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68301 00000100:00100000:3.0:1787826290.269588:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224053:x1874663332550272:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826290.269617:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826290.269644:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49e8b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801146 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.269720:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f49e92 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.269737:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f49e92 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.269749:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f49e92 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826290.269828:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x191:0x0]: returning size 4096 00000004:00200000:3.0:1787826290.269833:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826290.269876:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f49e8b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801146 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.269906:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f49e92 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801146 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.269916:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:3.0:1787826290.269920:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826290.269924:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826290.270075:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224053:x1874663332550272:12345-192.168.203.56@tcp:101:bash.0 Request processed in 489us (1378us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.270083:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68301 00000100:00100000:2.0:1787826290.274811:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826290.274993:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332550400 00000100:00100000:3.0:1787826290.275005:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332550400 00000100:00100000:3.0:1787826290.275095:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68302 00000100:00100000:3.0:1787826290.275136:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224053:x1874663332550400:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826290.275165:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826290.275191:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49e99 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080114d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.275267:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.275284:0:17813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49e4c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49e45 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.275294:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.275336:0:17813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49e4c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f49e45 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826290.275353:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346151680:0@lo:104: 00000100:00100000:3.0:1787826290.275414:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826290.275466:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753be40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826290.275704:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346151680 00000100:00100000:2.0:1787826290.275711:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346151680 00000100:00100000:2.0:1787826290.275756:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7831 00000100:00100000:2.0:1787826290.275765:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17813:x1874663346151680:12345-0@lo:104: 00010000:00010000:2.0:1787826290.275780:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.275875:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826290.275882:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17813:x1874663346151680:12345-0@lo:104: Request processed in 118us (466us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.275890:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7831 00010000:00010000:1.0:1787826290.275964:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826290.275973:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.275984:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efe00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826290.275991:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826290.276002:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.276037:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.276045:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826290.276076:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.276084:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f49e45 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f49e4c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.276147:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826290.276149:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826290.276151:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826290.276218:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346151680:0@lo:104: 00010000:00010000:0.0:1787826290.276248:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.276398:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346151808:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826290.276468:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.276560:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346151808 00000100:00100000:0.0:1787826290.276567:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346151808 00010000:00010000:0.0:1787826290.276593:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49e4c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f49e45 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826290.276629:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21886 00000100:00100000:0.0:1787826290.276638:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd82450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346151808:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826290.276650:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826290.279259:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f49e4c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f49e45 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.279289:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:0.0:1787826290.279295:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.279306:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.279316:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.279367:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826290.279470:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd82450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346151808:12345-0@lo:103:ldlm_bl.0 Request processed in 2831us (2999us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.279481:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21886 00000100:00100000:1.0:1787826290.279678:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346151808:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826290.279839:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.279854:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826290.279932:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18f:0x0]: returning size 4096 00000004:00200000:3.0:1787826290.279938:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826290.279997:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f49e99 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080114d expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.280018:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080114d expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.280027:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:3.0:1787826290.280030:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826290.280033:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826290.280143:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224053:x1874663332550400:12345-192.168.203.56@tcp:101:bash.0 Request processed in 5001us (5324us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.280152:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68302 00000100:00100000:0.0:1787826290.284649:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826290.288368:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332550656 00000100:00100000:2.0:1787826290.288387:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332550656 00000100:00100000:2.0:1787826290.288478:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68303 00000100:00100000:2.0:1787826290.288491:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224053:x1874663332550656:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826290.288518:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826290.288521:0:6529:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826290.288553:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f49ea0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080114d expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.288578:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:2.0:1787826290.288590:0:6529:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826290.288608:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49ea7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801154 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.288770:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.288785:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.288808:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.288824:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.288831:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.288843:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826290.289292:0:6529:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x51e:0x0], index 1: 0 00010000:00010000:2.0:1787826290.289464:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49ebc lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289476:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49ebc lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289485:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49ebc lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289529:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289537:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289545:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289552:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289559:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826290.289564:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289573:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289590:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f7952ca9e00) 00010000:00010000:2.0:1787826290.289593:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826290.289598:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289604:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49eae lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.289620:0:6529:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000441b031d reply state 000000001379c264 transno 8589957341 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289632:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289639:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f7952ca9000) 00010000:00010000:2.0:1787826290.289641:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826290.289644:0:6529:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826290.289683:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49ea7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801154 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289704:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49ebc lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801154 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.289713:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db7:0x0].0x0 (ffff9f7952ca9500) 00010000:00010000:2.0:1787826290.289716:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826290.289718:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826290.290351:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224053:x1874663332550656:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1855us (5710us total) trans 8589957341 rc 0/0 00000100:00100000:2.0:1787826290.290361:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68303 00000100:00100000:2.0:1787826290.293317:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826290.293413:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332550784 00000100:00100000:0.0:1787826290.293426:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332550784 00000100:00100000:0.0:1787826290.293526:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30795 00000100:00100000:0.0:1787826290.293542:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224053:x1874663332550784:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826290.293567:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826290.293595:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080115b expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826290.293627:0:8759:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x51e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787826290.293657:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=8102s v=(0 0) 00010000:00010000:0.0:1787826290.293663:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826290.293676:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 9 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.293696:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 9 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826290.293714:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826290.293808:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224053:x1874663332550784:12345-192.168.203.56@tcp:101:bash.0 Request processed in 270us (505us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.293816:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30795 00000100:00080000:2.0:1787826290.294059:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826290.300779:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826290.300872:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332551040 00000100:00100000:2.0:1787826290.300892:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332551040 00000100:00100000:2.0:1787826290.301004:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9317 00000100:00100000:2.0:1787826290.301035:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd3c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224053:x1874663332551040:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826290.301600:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd3c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224053:x1874663332551040:12345-192.168.203.56@tcp:35:bash.0 Request processed in 576us (831us total) trans 8589957342 rc 0/0 00000100:00100000:2.0:1787826290.301615:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9317 00000100:00100000:1.0:1787826290.309240:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826290.309868:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332551296 00000100:00100000:2.0:1787826290.309882:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332551296 00000100:00100000:2.0:1787826290.309963:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68304 00000100:00100000:2.0:1787826290.309975:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224053:x1874663332551296:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826290.310002:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826290.310006:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826290.310052:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f49e92 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801146 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.310079:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:2.0:1787826290.310085:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826290.310161:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f49ed1 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801162 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.310325:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.310340:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.310356:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.310372:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.310378:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.310385:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826290.310568:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b661880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346151936:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826290.310652:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826290.310699:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524050c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826290.311166:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346151936 00000100:00100000:3.0:1787826290.311176:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346151936 00000100:00100000:3.0:1787826290.311244:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23437 00000100:00100000:3.0:1787826290.311256:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6528:x1874663346151936:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787826290.311371:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6528:x1874663346151936:12345-0@lo:1000:mdt00_001.0 Request processed in 117us (718us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.311379:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23437 00000100:00100000:2.0:1787826290.311585:0:6528:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957340 gen 2 00000004:00080000:2.0:1787826290.311592:0:6528:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957340 committed transno 8589957340 00000100:00100000:2.0:1787826290.311630:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b661880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346151936:0@lo:1000:mdt00_001.0 00000004:00080000:2.0:1787826290.312553:0:6528:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4cb:0x0], index 0: 0 00010000:00010000:2.0:1787826290.314522:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49ee6 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.314553:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49ee6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.314569:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49ee6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.314620:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.314630:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.314637:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.314645:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.314651:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826290.316352:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.316378:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.316400:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:2.0:1787826290.316404:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826290.316410:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.316418:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49ed8 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826290.316443:0:6528:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009086fb00 reply state 00000000c24f9900 transno 4294985357 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.316457:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.316465:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x4ec94fd6 (ffff9f7952ca9e00) 00010000:00010000:2.0:1787826290.316468:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826290.316472:0:6528:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826290.316967:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f49ed1 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801162 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.316991:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49ee6 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801162 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.317000:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:2.0:1787826290.317002:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826290.317005:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826290.317154:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224053:x1874663332551296:12345-192.168.203.56@tcp:101:bash.0 Request processed in 7180us (7928us total) trans 4294985357 rc 0/0 00000100:00100000:2.0:1787826290.317164:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68304 00000100:00100000:3.0:1787826290.320344:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826290.321608:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332551424 00000100:00100000:2.0:1787826290.321627:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332551424 00000100:00100000:3.0:1787826290.321863:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30796 00000100:00100000:3.0:1787826290.322070:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224053:x1874663332551424:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826290.322100:0:35352:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826290.322250:0:35352:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801169 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826290.322285:0:35352:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4cb:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1787826290.322327:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cb:0x0].0x0 since=8102s v=(0 0) 00010000:00010000:3.0:1787826290.322333:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826290.322346:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 8 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.322366:0:35352:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 8 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826290.322381:0:35352:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826290.322498:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224053:x1874663332551424:12345-192.168.203.56@tcp:101:bash.0 Request processed in 624us (2167us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826290.322507:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30796 00000100:00080000:2.0:1787826290.322879:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826290.332485:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826290.332698:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332551680 00000100:00100000:1.0:1787826290.332711:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332551680 00000100:00100000:1.0:1787826290.332829:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9318 00000100:00100000:1.0:1787826290.332844:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224053:x1874663332551680:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:1.0:1787826290.333317:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224053:x1874663332551680:12345-192.168.203.56@tcp:35:bash.0 Request processed in 474us (843us total) trans 4294985358 rc 0/0 00000100:00100000:1.0:1787826290.333326:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9318 00000100:00100000:0.0:1787826290.356819:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826290.356927:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826290.357012:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826290.357046:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332551808 00000100:00100000:2.0:1787826290.357059:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332551808 00000100:00100000:2.0:1787826290.357169:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11034 00000100:00100000:2.0:1787826290.357183:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:34880:x1874663332551808:12345-192.168.203.56@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826290.357295:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332552064 00000100:00100000:1.0:1787826290.357300:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332552064 00000100:00100000:1.0:1787826290.357338:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11035 00000100:00100000:1.0:1787826290.357345:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:49150:x1874663332552064:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826290.357357:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826290.357398:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f49ee6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801162 expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.357422:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:1.0:1787826290.357441:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f49e84 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080113f expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.357452:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:1.0:1787826290.357460:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49c3f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e44:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010c1 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.357471:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e44:0x0].0x0 (ffff9f7961614600) 00010000:00010000:1.0:1787826290.357478:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826290.357538:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:49150:x1874663332552064:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 193us (611us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826290.357546:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11035 00010000:00010000:2.0:1787826290.357771:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826290.357788:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f49e5a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801131 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.357810:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826290.357820:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49ebc lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801154 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.357834:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db7:0x0].0x0 (ffff9f7952ca9500) 00010000:00010000:2.0:1787826290.357845:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49e76 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801138 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826290.357853:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7952ca9f00) 00010000:00010000:2.0:1787826290.357857:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826290.357891:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:34880:x1874663332551808:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 713us (1096us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.357897:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11034 00000100:00100000:1.0:1787826290.361275:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332551936 00000100:00100000:1.0:1787826290.361297:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332551936 00000100:00100000:1.0:1787826290.361431:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11036 00000100:00100000:1.0:1787826290.361444:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:6634:x1874663332551936:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826290.361464:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826290.361497:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f49cd2 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1db6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608010e4 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826290.361540:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db6:0x0].0x0 (ffff9f7961614b00) 00010000:00010000:1.0:1787826290.361555:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826290.361636:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:6634:x1874663332551936:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 194us (4622us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826290.361646:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11036 00000100:00100000:2.0:1787826290.674516:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346152192:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826290.674592:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346152320:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826290.674621:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.674662:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826290.674674:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826290.674681:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346152064:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826290.674685:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826290.674689:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346152448:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826290.674717:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826290.674717:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826290.674749:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346152192 00000100:00100000:2.0:1787826290.674749:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346152320 00000100:00100000:0.0:1787826290.674757:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346152192 00000100:00100000:2.0:1787826290.674757:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346152320 00000100:00100000:0.0:1787826290.674831:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346152064 00000100:00100000:0.0:1787826290.674834:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346152064 00000100:00100000:2.0:1787826290.675154:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6208 00000100:00100000:2.0:1787826290.675167:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346152192:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826290.675923:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6209 00000100:00100000:0.0:1787826290.675938:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7975894850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346152320:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787826290.675967:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826290.676053:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7975894850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346152320:12345-0@lo:13:osp-pre-0-1.0 Request processed in 121us (1389us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.676062:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6209 00000100:00100000:0.0:1787826290.676085:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6210 00000100:00100000:0.0:1787826290.676093:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346152064:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826290.676531:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826290.676587:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346152192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1422us (1964us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826290.676594:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6208 00000100:00100000:2.0:1787826290.676955:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346152192:0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787826290.677834:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826290.677861:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826290.677887:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346152576:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826290.677923:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346152064:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1828us (3202us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826290.677934:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6210 00000100:00100000:2.0:1787826290.677951:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826290.678219:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346152064:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826290.678502:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346152448 00000100:00100000:1.0:1787826290.678511:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346152448 00000100:00100000:1.0:1787826290.678560:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346152576 00000100:00100000:1.0:1787826290.678564:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346152576 00000100:00100000:1.0:1787826290.678592:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23438 00000100:00100000:1.0:1787826290.678603:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346152448:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826290.678645:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443422208 avail 1297182720 00000100:00100000:1.0:1787826290.678700:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346152448:12345-0@lo:41:osp-pre-1-0.0 Request processed in 100us (3983us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826290.678706:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23438 00000100:00100000:1.0:1787826290.678722:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23439 00000100:00100000:1.0:1787826290.678728:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346152576:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826290.678746:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442811904 avail 1296572416 00000100:00100000:1.0:1787826290.678806:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346152576:12345-0@lo:41:osp-pre-0-1.0 Request processed in 78us (857us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826290.678817:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23439 00000100:00100000:2.0:1787826290.685235:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346152576:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826290.686875:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346152448:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826290.686952:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346152320:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1787826290.863036:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826291.543591:0:165272:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:1.0:1787826291.543638:0:165272:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:0.0:1787826291.888044:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826292.908413:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826293.757184:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49d81 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826293.757207:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49d81 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.757257:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49d81 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.757265:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826293.757289:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f49d7a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49d81 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826293.757308:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985358 is committed 00000020:00080000:2.0:1787826293.757323:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784549fd80 refcount 2 committed 0 result 0 batchid 3401 00000020:00080000:2.0:1787826293.757326:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784549f300 00000020:00080000:2.0:1787826293.757330:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1504 00000020:00080000:2.0:1787826293.757332:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7851813000 00000020:00080000:2.0:1787826293.757335:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1185 00000001:00080000:2.0:1787826293.757911:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985355 is committed 00000020:00080000:2.0:1787826293.757919:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7852405180 refcount 2 committed 0 result 0 batchid 3400 00000020:00080000:2.0:1787826293.757923:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852405d80 00000020:00080000:2.0:1787826293.757927:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1503 00000020:00080000:2.0:1787826293.757929:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e774a00 00000020:00080000:2.0:1787826293.757932:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1184 00000004:00080000:2.0:1787826293.757943:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 400 committed 00000020:00080000:2.0:1787826293.757957:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784753b180 refcount 2 committed 0 result 0 batchid 3399 00000020:00080000:2.0:1787826293.757960:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2def00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784753b600 00000020:00080000:2.0:1787826293.757963:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1502 00000020:00080000:2.0:1787826293.757965:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2dec00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7847b17000 00000020:00080000:2.0:1787826293.757968:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1183 00000020:00080000:2.0:1787826293.757975:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7852982600 refcount 2 committed 0 result 0 batchid 3398 00000020:00080000:2.0:1787826293.757978:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78448e0900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784aa28cc0 00000020:00080000:2.0:1787826293.757980:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1501 00000020:00080000:2.0:1787826293.757982:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f794b603e00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784ae9dc00 00000020:00080000:2.0:1787826293.757985:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1182 00000001:00080000:2.0:1787826293.757995:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985349 is committed 00000020:00080000:2.0:1787826293.758004:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3397 batchid updated 00010000:00010000:2.0:1787826293.758152:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758162:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758169:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758175:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826293.758179:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758187:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758201:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7952ca9d00) 00010000:00010000:2.0:1787826293.758204:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826293.758207:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758213:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f49e37 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758231:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758237:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758242:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758246:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826293.758249:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758254:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758262:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7952ca9400) 00010000:00010000:2.0:1787826293.758264:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826293.758267:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758272:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f49e3e lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758290:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758296:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758302:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758306:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826293.758309:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758314:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758321:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x4ec94fd6 (ffff9f7952ca9e00) 00010000:00010000:2.0:1787826293.758324:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826293.758327:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.758332:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f49edf lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826293.758964:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346152704:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787826293.759305:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:2.0:1787826293.759808:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3401 committed batchid 3397 00000004:00080000:2.0:1787826293.759916:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784c4f4180 with version 6800 00000020:00080000:2.0:1787826293.759934:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3394 cancel update log [0x200000401:0x3:0x0].1497: rc = 0 00000020:00080000:2.0:1787826293.759978:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3394 cancel update log [0x240000400:0x3:0x0].1178: rc = 0 00000004:00080000:2.0:1787826293.760002:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784c4f4d80 with version 6801 00000020:00080000:2.0:1787826293.760005:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3395 cancel update log [0x200000401:0x3:0x0].1498: rc = 0 00000020:00080000:2.0:1787826293.760040:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3395 cancel update log [0x240000400:0x3:0x0].1179: rc = 0 00000004:00080000:2.0:1787826293.760061:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7851057780 with version 6802 00000020:00080000:2.0:1787826293.760064:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3396 cancel update log [0x200000401:0x3:0x0].1499: rc = 0 00000020:00080000:2.0:1787826293.760084:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3396 cancel update log [0x240000400:0x3:0x0].1180: rc = 0 00000001:00080000:1.0:1787826293.760501:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973840 is committed 00000001:00080000:1.0:1787826293.760512:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973839 is committed 00000001:00080000:1.0:1787826293.760520:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973838 is committed 00000004:00080000:2.0:1787826293.760614:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78510579c0 with version 6803 00000020:00080000:2.0:1787826293.760619:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3397 cancel update log [0x200000401:0x3:0x0].1500: rc = 0 00000004:00080000:0.0:1787826293.760695:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6800 rpc_version 6800 00000020:00100000:0.0:1787826293.760727:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826293.760737:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784836f100 x1874663346152960/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826293.760764:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346152960:0@lo:1000:osp_up0-1.0 00000020:00080000:2.0:1787826293.760795:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3397 cancel update log [0x240000400:0x3:0x0].1181: rc = 0 00000020:00080000:2.0:1787826293.760803:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3397 00000100:00100000:0.0:1787826293.760827:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826293.760869:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826293.760881:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346152704 00000100:00100000:0.0:1787826293.760884:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346153088:0@lo:6:osp-syn-1-1.0 00000100:00100000:2.0:1787826293.760887:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346152704 00000100:00100000:0.0:1787826293.760909:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826293.760933:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11037 00000100:00100000:2.0:1787826293.760943:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346152704:12345-0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826293.760944:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346153088 00000100:00100000:0.0:1787826293.760947:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346153088 00010000:00010000:2.0:1787826293.760956:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826293.760980:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f49d81 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18e:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49d7a expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826293.760982:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30797 00010000:00010000:2.0:1787826293.760997:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18e:0x0].0x5a734ce9 (ffff9f794b57a000) 00010000:00010000:2.0:1787826293.761008:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826293.761145:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346152960 00000100:00100000:1.0:1787826293.761151:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346152960 00000100:00100000:1.0:1787826293.761179:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23440 00000100:00100000:1.0:1787826293.761185:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346152960:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826293.761441:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346152960:12345-0@lo:1000:osp_up0-1.0 Request processed in 255us (613us total) trans 8589957343 rc 0/0 00000100:00100000:1.0:1787826293.761449:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23440 00000100:00080000:1.0:1787826293.761489:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826293.761543:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346152960:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826293.761553:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6801 rpc_version 6801 00000020:00100000:1.0:1787826293.761566:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826293.761571:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f79785e4a80 x1874663346153344/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826293.761583:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346153344:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826293.761611:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826293.761629:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346153088:12345-0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1787826293.761638:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00002000:00080000:0.0:1787826293.761648:0:35097:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1224 count 1 00000100:00100000:1.0:1787826293.761652:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346153344 00000100:00100000:1.0:1787826293.761655:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346153344 00010000:00010000:0.0:1787826293.761669:0:35097:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826293.761689:0:35097:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 8 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826293.761697:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4c8:0x0].0x0 since=9s v=(0 0) 00010000:00010000:0.0:1787826293.761701:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826293.761705:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826293.761729:0:35097:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 8 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826293.761742:0:35097:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 8 pid: 17826 timeout: 8205 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826293.761758:0:35097:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:35097:1874663346153472:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826293.761807:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23441 00000100:00100000:1.0:1787826293.761816:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346153344:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826293.761823:0:35097:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826293.761911:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346153344:12345-0@lo:1000:osp_up0-1.0 Request processed in 96us (299us total) trans 8589957344 rc 0/0 00000100:00100000:1.0:1787826293.761917:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23441 00000100:00100000:2.0:1787826293.763002:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346152704:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 2055us (3692us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826293.763016:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11037 00000100:00100000:1.0:1787826293.763187:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346152704:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826293.763558:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346153344:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826293.763572:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6802 rpc_version 6802 00000020:00100000:0.0:1787826293.763590:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826293.763596:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784836d880 x1874663346153728/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826293.763615:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346153728:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826293.763658:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826293.763687:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826293.763764:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346153728 00000100:00100000:1.0:1787826293.763780:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346153728 00000100:00100000:1.0:1787826293.763824:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23442 00000100:00100000:1.0:1787826293.763832:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346153728:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826293.763982:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346153728:12345-0@lo:1000:osp_up0-1.0 Request processed in 150us (323us total) trans 8589957345 rc 0/0 00000100:00100000:1.0:1787826293.763990:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23442 00000100:00100000:2.0:1787826293.764601:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346153728:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826293.764612:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6803 rpc_version 6803 00000020:00100000:2.0:1787826293.764639:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826293.764645:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5a7b80 x1874663346153984/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826293.764662:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a7b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346153984:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826293.764835:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826293.764862:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826293.764904:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346153984 00000100:00100000:1.0:1787826293.764910:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346153984 00000100:00100000:1.0:1787826293.764953:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23443 00000100:00100000:1.0:1787826293.764962:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346153984:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826293.765094:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346153984:12345-0@lo:1000:osp_up0-1.0 Request processed in 132us (257us total) trans 8589957346 rc 0/0 00000100:00100000:1.0:1787826293.765101:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23443 00000100:00100000:0.0:1787826293.765151:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a7b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346153984:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826293.773204:0:35097:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:35097:1874663346153472:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826293.773223:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:2.0:1787826293.773237:0:35097:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826293.774627:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332552192 00000100:00100000:3.0:1787826293.774640:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332552192 00010000:00010000:3.0:1787826293.775084:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 9 pid: 17826 timeout: 8205 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826293.775149:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21887 00000100:00100000:3.0:1787826293.775159:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332552192:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826293.775175:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826293.775182:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 9 pid: 17826 timeout: 8205 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826293.775198:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 9 pid: 17826 timeout: 8205 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826293.775221:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f49b97 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801082 expref: 9 pid: 17826 timeout: 8205 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826293.775237:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826293.775241:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826293.775247:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826293.775258:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826293.775285:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00080000:0.0:1787826293.775737:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826293.775905:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332552192:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 745us (2679us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826293.776058:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21887 00010000:00010000:0.0:1787826293.777210:0:35097:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826293.777229:0:35097:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826293.777243:0:35097:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826293.777252:0:35097:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826293.777306:0:35097:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826293.777483:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346153088:12345-0@lo:6:osp-syn-1-1.0 Request processed in 16491us (16569us total) trans 4294976149 rc 0/0 00000100:00100000:0.0:1787826293.777492:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30797 00000004:00080000:0.0:1787826293.777555:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7850ad6a00/2, rc 0, transno 4294976149 00000100:00100000:0.0:1787826293.777561:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346153088:0@lo:6:osp-syn-1-1.0 00010000:00010000:1.0:1787826293.779614:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826293.779627:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.779641:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826293.779645:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.779719:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.779743:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.779751:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008837638e/0xe21309aa81f49efb lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.779780:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826293.779782:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826293.779784:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826293.935101:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826293.938474:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826293.949937:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332552320 00000100:00100000:1.0:1787826293.949953:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332552320 00000100:00100000:1.0:1787826293.950038:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68305 00000100:00100000:1.0:1787826293.950049:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224234:x1874663332552320:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826293.950072:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826293.950093:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f02 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801170 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.950591:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.950608:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.950624:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826293.950667:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826293.950671:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826293.950688:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.950714:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f02 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.950728:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.950737:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826293.950740:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826293.950744:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826293.950830:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224234:x1874663332552320:12345-192.168.203.56@tcp:101:cat.0 Request processed in 783us (12386us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.950837:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68305 00000100:00080000:2.0:1787826293.951321:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826293.957386:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826293.958903:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332552448 00000100:00100000:1.0:1787826293.958916:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332552448 00000100:00100000:1.0:1787826293.959070:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68306 00000100:00100000:1.0:1787826293.959083:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224234:x1874663332552448:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826293.959120:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826293.959142:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f10 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801177 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959213:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959228:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959239:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959254:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959262:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959269:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959388:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49f25 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959397:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49f25 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959404:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49f25 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826293.959425:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49f25 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959439:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959446:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959452:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959458:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959475:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826293.959480:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959487:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959500:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826293.959503:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826293.959508:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.959514:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f49f17 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.961868:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.961883:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.961890:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.961896:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826293.961901:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.961908:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.961922:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1600) 00010000:00010000:1.0:1787826293.961925:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826293.961928:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.961935:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f49f1e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.962063:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f10 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801177 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.962079:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49f25 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801177 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.962087:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826293.962089:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826293.962091:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826293.962368:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224234:x1874663332552448:12345-192.168.203.56@tcp:101:cat.0 Request processed in 3285us (4990us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.962377:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68306 00000100:00100000:1.0:1787826293.970198:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826293.970300:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332552576 00000100:00100000:1.0:1787826293.970310:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332552576 00000100:00100000:1.0:1787826293.970385:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68307 00000100:00100000:1.0:1787826293.970398:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224234:x1874663332552576:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826293.970422:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826293.970447:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49f2c lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080117e expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.970836:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49f33 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.970852:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49f33 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.970866:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49f33 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826293.970998:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e46:0x0]: returning size 4096 00000004:00200000:1.0:1787826293.971004:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826293.971457:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49f2c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080117e expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.971550:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49f33 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080117e expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.971560:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1800) 00010000:00010000:1.0:1787826293.971563:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826293.971567:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826293.971634:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224234:x1874663332552576:12345-192.168.203.56@tcp:101:cat.0 Request processed in 1239us (1444us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.971643:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68307 00000100:00080000:3.0:1787826293.972369:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826293.979063:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826293.979742:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332552704 00000100:00100000:1.0:1787826293.979753:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332552704 00000100:00100000:1.0:1787826293.979825:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68308 00000100:00100000:1.0:1787826293.979837:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224234:x1874663332552704:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826293.979856:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826293.979878:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49f3a lrc: 2/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801185 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.979933:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.979946:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.979957:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826293.980190:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x191:0x0]: returning size 4096 00000004:00200000:1.0:1787826293.980196:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826293.980234:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f49f3a lrc: 0/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.980252:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826293.980261:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826293.980263:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826293.980268:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:1.0:1787826293.980332:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224234:x1874663332552704:12345-192.168.203.56@tcp:101:cat.0 Request processed in 496us (1276us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.980339:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68308 00000100:00100000:2.0:1787826293.982621:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.982662:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332552960 00000100:00100000:3.0:1787826293.982672:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332552960 00000100:00100000:3.0:1787826293.982733:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14255 00000100:00100000:3.0:1787826293.982744:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2396:x1874663332552960:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826293.982934:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2396:x1874663332552960:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 191us (320us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826293.982942:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14255 00000100:00100000:3.0:1787826293.983090:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.983224:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.983248:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.983312:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.983332:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.983348:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.983365:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.983394:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826293.983417:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332553472 00000100:00100000:3.0:1787826293.983423:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332553472 00000100:00100000:3.0:1787826293.983459:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332553344 00000100:00100000:1.0:1787826293.983585:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332553216 00000100:00100000:1.0:1787826293.983589:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332553216 00000100:00100000:1.0:1787826293.983621:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332553728 00000100:00100000:1.0:1787826293.983624:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332553728 00000100:00100000:1.0:1787826293.983639:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332553088 00000100:00100000:1.0:1787826293.983641:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332553088 00000100:00100000:1.0:1787826293.983656:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332553600 00000100:00100000:1.0:1787826293.983659:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332553600 00000100:00100000:1.0:1787826293.983677:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8782 00000100:00100000:1.0:1787826293.983685:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796eda5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332553216:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826293.983729:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796eda5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332553216:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 44us (640us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.983735:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8782 00000100:00100000:1.0:1787826293.983915:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8783 00000100:00100000:1.0:1787826293.983923:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332553728:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826293.983950:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332553728:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (701us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.983957:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8783 00000100:00100000:1.0:1787826293.984559:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8784 00000100:00100000:1.0:1787826293.984574:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332553088:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826293.984613:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332553088:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 44us (1279us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.984623:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8784 00000100:00100000:1.0:1787826293.984641:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8785 00000100:00100000:1.0:1787826293.984647:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332553600:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826293.984673:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332553600:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (1308us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.984679:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8785 00000100:00100000:2.0:1787826293.985234:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332553856 00000100:00100000:2.0:1787826293.985244:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332553856 00000100:00100000:2.0:1787826293.985289:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14256 00000100:00100000:2.0:1787826293.985298:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332553472:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826293.985339:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332553472:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 43us (2115us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826293.985346:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14256 00000100:00100000:2.0:1787826293.985362:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68309 00000100:00100000:2.0:1787826293.985369:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224234:x1874663332553856:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826293.985380:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826293.985402:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49f48 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080118c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.985547:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.985563:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.985574:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826293.985676:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18f:0x0]: returning size 4096 00000004:00200000:2.0:1787826293.985681:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826293.985719:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49f48 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.985739:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826293.985748:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f784abfeb00) 00010000:00010000:2.0:1787826293.985751:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826293.985754:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826293.985793:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224234:x1874663332553856:12345-192.168.203.56@tcp:101:cat.0 Request processed in 424us (2398us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826293.985800:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68309 00000100:00100000:1.0:1787826293.987227:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332552832 00000100:00100000:1.0:1787826293.987239:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332552832 00000100:00100000:1.0:1787826293.987289:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2214 00000100:00100000:1.0:1787826293.987299:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf43050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332552832:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826293.987358:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf43050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332552832:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 60us (4045us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826293.987365:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2214 00000100:00080000:2.0:1787826293.988796:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826293.988805:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826293.988809:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826293.988938:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332553344 00000100:00100000:3.0:1787826293.989547:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14257 00000100:00100000:3.0:1787826293.989559:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332553344:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826293.989631:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332553344:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 75us (6281us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826293.989639:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14257 00000100:00080000:2.0:1787826293.990562:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826294.001695:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.001790:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332554112 00000100:00100000:2.0:1787826294.001800:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332554112 00000100:00100000:2.0:1787826294.001858:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68310 00000100:00100000:2.0:1787826294.001868:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224234:x1874663332554112:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826294.001890:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826294.001911:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49f56 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801193 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.001988:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.002002:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.002012:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.002022:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.002029:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.002035:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:2.0:1787826294.002103:0:6527:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000100:00100000:1.0:1787826294.439619:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.439837:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332554240 00000100:00100000:2.0:1787826294.439857:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332554240 00000100:00100000:2.0:1787826294.439956:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68311 00000100:00100000:2.0:1787826294.439968:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224236:x1874663332554240:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826294.439989:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826294.440010:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49f6b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080119a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.440077:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.440089:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.440099:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826294.440543:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826294.440547:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826294.440570:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.440597:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f49f6b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.440617:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.440624:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826294.440627:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826294.440631:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:2.0:1787826294.440698:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224236:x1874663332554240:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 732us (1088us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.440705:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68311 00000100:00100000:3.0:1787826294.446809:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826294.452060:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332554368 00000100:00100000:1.0:1787826294.452075:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332554368 00000100:00100000:2.0:1787826294.452359:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68312 00000100:00100000:2.0:1787826294.452373:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224236:x1874663332554368:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826294.452393:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826294.452413:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49f79 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011a1 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452482:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452495:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452506:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452556:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452564:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452570:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452715:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452728:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452741:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826294.452760:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452782:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452788:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452794:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452799:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452805:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826294.452809:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452816:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452828:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826294.452831:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826294.452836:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452842:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49f80 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452865:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452870:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452875:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452879:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826294.452882:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452887:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452894:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe100) 00010000:00010000:2.0:1787826294.452895:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826294.452898:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452903:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49f87 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452939:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f49f79 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452953:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.452964:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:2.0:1787826294.452967:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826294.452969:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826294.453036:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224236:x1874663332554368:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 666us (6237us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.453043:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68312 00000100:00100000:3.0:1787826294.457915:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826294.460910:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332554496 00000100:00100000:0.0:1787826294.460927:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332554496 00000100:00100000:0.0:1787826294.461004:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68313 00000100:00100000:0.0:1787826294.461016:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:224236:x1874663332554496:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826294.461264:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826294.461287:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49f95 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011a8 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.461361:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f49f9c lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.461373:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f49f9c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.461385:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f49f9c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826294.461477:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e46:0x0]: returning size 4096 00000004:00200000:0.0:1787826294.461483:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826294.461520:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f49f95 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011a8 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.461544:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f49f9c lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011a8 expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.461551:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1800) 00010000:00010000:0.0:1787826294.461775:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826294.461781:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826294.461853:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224236:x1874663332554496:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 838us (3948us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826294.461861:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68313 00000100:00100000:0.0:1787826294.468882:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.469480:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332554624 00000100:00100000:2.0:1787826294.469491:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332554624 00000100:00100000:2.0:1787826294.469563:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68314 00000100:00100000:2.0:1787826294.469574:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224236:x1874663332554624:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826294.469595:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826294.469615:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49fa3 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011af expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469681:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 3/1,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469694:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469704:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469718:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469726:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469733:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469826:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49fb8 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469834:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49fb8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469840:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49fb8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826294.469880:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e47:0x0]: returning size 6 00000004:00200000:2.0:1787826294.469884:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826294.469903:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49fb8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469915:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49fb8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469925:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469931:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469937:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469944:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469953:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826294.469958:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469965:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469976:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:2.0:1787826294.469979:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826294.469982:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 3/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.469988:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f49faa lrc: 0/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470001:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470008:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470014:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470019:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826294.470022:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470028:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470039:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x4ec94fd6 (ffff9f784abfe100) 00010000:00010000:2.0:1787826294.470041:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826294.470044:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470050:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f49fb1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470094:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f49fa3 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011af expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470169:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49fb8 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011af expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.470178:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f784abfe200) 00010000:00010000:2.0:1787826294.470180:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826294.470183:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826294.470241:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224236:x1874663332554624:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 668us (1369us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.470249:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68314 00000100:00100000:3.0:1787826294.477851:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.477976:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332554752 00000100:00100000:2.0:1787826294.477985:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332554752 00000100:00100000:2.0:1787826294.478050:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68315 00000100:00100000:2.0:1787826294.478059:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224236:x1874663332554752:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826294.478083:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826294.478133:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fbf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011b6 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478251:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478266:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478277:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478287:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478293:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478299:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478373:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49fd4 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478382:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49fd4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478389:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49fd4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826294.478427:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db7:0x0]: returning size 6 00000004:00200000:2.0:1787826294.478432:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826294.478450:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49fd4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478465:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49fd4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478474:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478480:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478487:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478493:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478499:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826294.478502:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478509:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478521:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f784abfeb00) 00010000:00010000:2.0:1787826294.478523:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826294.478527:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478533:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f49fc6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478545:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478939:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478949:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478954:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826294.478957:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478964:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478972:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f7952ca9000) 00010000:00010000:2.0:1787826294.478974:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826294.478977:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.478983:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f49fcd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.479010:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fbf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011b6 expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.479022:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49fd4 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011b6 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.479418:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db7:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:2.0:1787826294.479421:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826294.479424:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826294.479518:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224236:x1874663332554752:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1458us (1675us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.479526:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68315 00000100:00100000:1.0:1787826294.485797:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826294.485873:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826294.485956:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332554880 00000100:00100000:0.0:1787826294.485967:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332554880 00000100:00100000:0.0:1787826294.486165:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11038 00000100:00100000:0.0:1787826294.486176:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332554880:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:0.0:1787826294.486191:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826294.486217:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f49fb8 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011af expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.486235:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f784abfe200) 00010000:00010000:0.0:1787826294.486246:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826294.486313:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332554880:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 139us (438us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826294.486321:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11038 00000100:00100000:2.0:1787826294.488021:0:6540:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332555136 00000100:00100000:2.0:1787826294.488048:0:6540:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332555136 00000100:00100000:2.0:1787826294.488160:0:6540:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1226 00000100:00100000:2.0:1787826294.488172:0:6540:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224236:x1874663332555136:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:2.0:1787826294.488226:0:6540:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826294.488252:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f49fd4 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011b6 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.488273:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db7:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:2.0:1787826294.488721:0:6540:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826294.488815:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154112:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.488883:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.488916:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826294.489011:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346154112 00000100:00100000:1.0:1787826294.489018:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346154112 00000100:00100000:1.0:1787826294.489085:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23444 00000100:00100000:1.0:1787826294.489094:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154112:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.489235:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154112:12345-0@lo:1000:mdt_io00_000.0 Request processed in 142us (351us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.489246:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23444 00000100:00100000:1.0:1787826294.489280:0:6540:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985349 gen 2 00000004:00080000:1.0:1787826294.489601:0:6540:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985348 committed transno 4294985349 00000020:00080000:1.0:1787826294.489607:0:6540:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f784753b480 refcount 2 committed 0 result 0 batchid 4294979078 00000020:00080000:1.0:1787826294.489612:0:6540:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2de800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784753b6c0 00000020:00080000:1.0:1787826294.489616:0:6540:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].934 00000020:00080000:1.0:1787826294.489618:0:6540:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2de300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e74e600 00000020:00080000:1.0:1787826294.489620:0:6540:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].931 00000004:00080000:1.0:1787826294.489648:0:6540:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985349 committed transno 4294985349 00000100:00100000:1.0:1787826294.489665:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154112:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.489726:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154240:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.489759:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.489786:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa840 going to sleep for 11 seconds 00000020:00080000:1.0:1787826294.489798:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979078 committed batchid 4294979077 00000020:00080000:1.0:1787826294.489868:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979077 00000100:00100000:1.0:1787826294.489882:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346154240 00000100:00100000:1.0:1787826294.489886:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346154240 00000100:00100000:1.0:1787826294.489916:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23445 00000100:00100000:1.0:1787826294.489923:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154240:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.489986:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154240:12345-0@lo:1000:mdt_io00_000.0 Request processed in 63us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.489993:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23445 00000100:00100000:1.0:1787826294.490023:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154240:0@lo:1000:mdt_io00_000.0 00000004:00080000:1.0:1787826294.490512:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x191:0x0] 00000100:00100000:1.0:1787826294.490546:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154368:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.490573:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.490601:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.492867:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346154368 00000100:00100000:2.0:1787826294.492883:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346154368 00000100:00100000:2.0:1787826294.492944:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23446 00000100:00100000:2.0:1787826294.492954:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154368:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.493397:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154368:12345-0@lo:1000:mdt_io00_000.0 Request processed in 442us (2820us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.493404:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23446 00000100:00100000:1.0:1787826294.493837:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154368:0@lo:1000:mdt_io00_000.0 00000004:00080000:1.0:1787826294.493877:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e46:0x0] 00000100:00100000:1.0:1787826294.493933:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154496:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826294.494510:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826294.494541:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826294.494565:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332555392 00000100:00100000:3.0:1787826294.494571:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332555392 00000100:00100000:3.0:1787826294.494658:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30798 00000100:00100000:3.0:1787826294.494667:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332555392:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:3.0:1787826294.494755:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332555392:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 88us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.494763:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30798 00000100:00100000:1.0:1787826294.495442:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.495485:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826294.495509:0:24158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332555264 00000100:00100000:1.0:1787826294.495516:0:24158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332555264 00002000:00100000:1.0:1787826294.495554:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x51e:0x0] (0->5) req@ffff9f784f685c00 x1874663332555264/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:650/0 lens 496/0 e 0 to 0 dl 1787826305 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826294.495565:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f685c00 with x1874663332555264 ext(0->5) 00002000:00010000:1.0:1787826294.495571:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784f685c00 00000100:00100000:1.0:1787826294.495595:0:24158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27636 00000100:00100000:1.0:1787826294.495604:0:24158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332555264:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826294.496574:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346154496 00000100:00100000:3.0:1787826294.496583:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346154496 00000100:00100000:3.0:1787826294.496631:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23447 00000100:00100000:3.0:1787826294.496639:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154496:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826294.496954:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154496:12345-0@lo:1000:mdt_io00_000.0 Request processed in 314us (1513us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.496961:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23447 00000100:00100000:0.0:1787826294.499235:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154496:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826294.499375:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x191:0x0] 00000100:00100000:0.0:1787826294.499431:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154624:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.499498:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.499537:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.499733:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346154624 00000100:00100000:2.0:1787826294.499742:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346154624 00000100:00100000:2.0:1787826294.499809:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23448 00000100:00100000:2.0:1787826294.499821:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154624:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.499912:0:24158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2395:x1874663332555264:12345-192.168.203.56@tcp:4:bash.0 Request processed in 4306us (5366us total) trans 4294976150 rc 0/0 00000100:00100000:2.0:1787826294.499917:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154624:12345-0@lo:1000:mdt_io00_000.0 Request processed in 97us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.499925:0:24158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27636 00000100:00100000:2.0:1787826294.499928:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23448 00002000:00100000:1.0:1787826294.499940:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x51e:0x0] (0->5) req@ffff9f784f685c00 x1874663332555264/t4294976150(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:650/0 lens 496/448 e 0 to 0 dl 1787826305 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826294.499951:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f685c00 with x1874663332555264 ext(0->5) 00002000:00010000:1.0:1787826294.499958:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784f685c00 00000100:00100000:0.0:1787826294.500707:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154624:0@lo:1000:mdt_io00_000.0 00000004:00020000:0.0:1787826294.500756:0:6540:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00000004:00020000:2.0:1787826294.500793:0:6527:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4501 00010000:00010000:0.0:1787826294.500818:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 3/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.500827:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 3/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.500836:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 3/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.500843:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826294.500850:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154752:0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826294.500855:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.500866:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826294.500876:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db7:0x0]: returning size 6 00000100:00100000:0.0:1787826294.500889:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.500914:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00010000:00010000:2.0:1787826294.500952:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.500959:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.500975:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.500983:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.500989:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826294.500994:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501001:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826294.501013:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346154752 00010000:00010000:2.0:1787826294.501014:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f784abfeb00) 00010000:00010000:2.0:1787826294.501016:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826294.501019:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346154752 00010000:00010000:2.0:1787826294.501021:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501039:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f49f5d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826294.501051:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23449 00010000:00010000:2.0:1787826294.501054:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826294.501059:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346154752:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826294.501061:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501066:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.501069:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826294.501071:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826294.501082:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49fdb expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501083:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501089:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501097:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f7952ca9000) 00010000:00010000:2.0:1787826294.501099:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826294.501101:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.501103:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501132:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f49f64 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.501136:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 4/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49fdb expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501179:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f49f56 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501192:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.501198:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db7:0x0].0x0 (ffff9f784abfef00) 00010000:00010000:2.0:1787826294.501200:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826294.501202:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826294.501242:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224234:x1874663332554112:12345-192.168.203.56@tcp:101:cat.0 Request processed in 499376us (499567us total) trans 8589957347 rc 0/0 00000100:00100000:2.0:1787826294.501249:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68310 00010000:00010000:3.0:1787826294.501299:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.501381:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 6 pid: 35098 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826294.501392:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346154880:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826294.501433:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bf00 going to sleep for 11 seconds 00000001:00080000:0.0:1787826294.503350:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976150 is committed 00000001:00080000:0.0:1787826294.503377:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976149 is committed 00000100:00100000:0.0:1787826294.503983:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826294.506557:0:20162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332555520 00000100:00100000:3.0:1787826294.506578:0:20162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332555520 00002000:00100000:3.0:1787826294.506659:0:20162:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4cb:0x0] (0->5) req@ffff9f784adf9500 x1874663332555520/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:650/0 lens 496/0 e 0 to 0 dl 1787826305 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826294.506673:0:20162:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784adf9500 with x1874663332555520 ext(0->5) 00002000:00010000:3.0:1787826294.506680:0:20162:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784adf9500 00000100:00100000:3.0:1787826294.506728:0:20162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27637 00000100:00100000:3.0:1787826294.506742:0:20162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332555520:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826294.510175:0:20162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332555520:12345-192.168.203.56@tcp:4:bash.0 Request processed in 3418us (6181us total) trans 4294973841 rc 0/0 00000100:00100000:3.0:1787826294.510193:0:20162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27637 00002000:00100000:3.0:1787826294.510316:0:20162:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4cb:0x0] (0->5) req@ffff9f784adf9500 x1874663332555520/t4294973841(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:650/0 lens 496/448 e 0 to 0 dl 1787826305 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826294.510330:0:20162:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784adf9500 with x1874663332555520 ext(0->5) 00002000:00010000:3.0:1787826294.510338:0:20162:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784adf9500 00000100:00100000:1.0:1787826294.510929:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.511184:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332555648 00000100:00100000:2.0:1787826294.511198:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332555648 00000100:00100000:2.0:1787826294.511427:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30799 00000100:00100000:2.0:1787826294.511441:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332555648:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:1.0:1787826294.515868:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826294.515933:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.516319:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346154880:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826294.516329:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332555776 00000100:00100000:0.0:1787826294.516340:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332555776 00010000:00010000:2.0:1787826294.516370:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49fdb expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.516386:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:2.0:1787826294.516391:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826294.516392:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 7 pid: 35098 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.516397:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49fdb expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.516418:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:0.0:1787826294.516456:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21888 00000100:00100000:0.0:1787826294.516470:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd81c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332555776:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826294.516493:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787826294.516499:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346154752:12345-0@lo:101:mdt_io00_000.0 Request processed in 15439us (15607us total) trans 0 rc 0/0 00010000:00010000:0.0:1787826294.516502:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 7 pid: 35098 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826294.516513:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23449 00010000:00010000:0.0:1787826294.516524:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 7 pid: 35098 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.516551:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f49f41 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801185 expref: 7 pid: 35098 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.516571:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:0.0:1787826294.516577:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49fdb expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.516593:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49fdb expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.516602:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f49fdb expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.516640:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49fdb expref: 5 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826294.516657:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:17812:1874663346155008:0@lo:105: 00000100:00100000:2.0:1787826294.516697:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346154752:0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826294.516718:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 4/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.516730:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 4/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826294.517297:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.517338:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826294.517601:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332555904 00000100:00100000:3.0:1787826294.517611:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332555904 00000100:00100000:1.0:1787826294.517625:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826294.517643:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346155008 00000100:00100000:1.0:1787826294.517648:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346155008 00000100:00100000:1.0:1787826294.517701:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7832 00000100:00100000:3.0:1787826294.517714:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30800 00000100:00100000:1.0:1787826294.517715:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7855266c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:17812:x1874663346155008:12345-0@lo:105: 00000100:00100000:3.0:1787826294.517728:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332555904:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826294.517731:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 5/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.517746:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 5/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.517750:0:8444:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826294.517776:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 5/0,1 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.517782:0:8444:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000027ed3819/0xe21309aa81f49ff7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011bd expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.517788:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.517805:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.517815:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.517836:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43000) 00000100:00100000:3.0:1787826294.517841:0:8444:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8444:1874663346155136:192.168.203.56@tcp:106: 00000100:00100000:3.0:1787826294.517919:0:8444:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b3c0 going to sleep for 11 seconds 00010000:00010000:2.0:1787826294.517926:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.517948:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826294.517981:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x191:0x0] 00010000:00010000:2.0:1787826294.518024:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.518033:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.518045:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826294.518064:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346155264:0@lo:101:mdt_io00_000.0 00000100:00100000:1.0:1787826294.518348:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7855266c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:17812:x1874663346155008:12345-0@lo:105: Request processed in 636us (1052us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.518356:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7832 00000100:00100000:1.0:1787826294.518464:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:17812:1874663346155008:0@lo:105: 00010000:00010000:1.0:1787826294.518478:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826294.518507:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd81c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332555776:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2039us (2655us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.518515:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21888 00002000:00010000:3.0:1787826294.522313:0:8444:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x2c0000401:0x51e:0x0] updating lvb size: 0 -> 6 00002000:00010000:3.0:1787826294.522324:0:8444:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x2c0000401:0x51e:0x0] updating lvb mtime: 0 -> 1787826289 00002000:00010000:3.0:1787826294.522327:0:8444:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x2c0000401:0x51e:0x0] updating lvb atime: 0 -> 1787826289 00002000:00010000:3.0:1787826294.522330:0:8444:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x2c0000401:0x51e:0x0] updating lvb ctime: 0 -> 1787826289 00002000:00010000:3.0:1787826294.522331:0:8444:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x2c0000401:0x51e:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:3.0:1787826294.522361:0:8444:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8444:1874663346155136:192.168.203.56@tcp:106: 00010000:00010000:3.0:1787826294.522407:0:8444:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000027ed3819/0xe21309aa81f49ff7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011bd expref: 6 pid: 8444 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.522435:0:8444:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000027ed3819/0xe21309aa81f49ff7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011bd expref: 6 pid: 8444 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.522451:0:8444:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826294.523341:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332555904:12345-192.168.203.56@tcp:101:cat.0 Request processed in 5607us (7397us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826294.523354:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30800 00000100:00100000:2.0:1787826294.525436:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.525497:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.525531:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346155264 00000100:00100000:2.0:1787826294.525542:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346155264 00000100:00100000:2.0:1787826294.525613:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23450 00000100:00100000:2.0:1787826294.525626:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346155264:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826294.525653:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826294.525684:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4a005 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f49ffe expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.525720:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4a005 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f49ffe expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.525738:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4a005 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f49ffe expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.525748:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x4ec94fd6 (ffff9f784abfed00) 00010000:00010000:2.0:1787826294.525752:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826294.525757:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826294.525819:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346155264:12345-0@lo:101:mdt_io00_000.0 Request processed in 195us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.525826:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23450 00000100:00100000:2.0:1787826294.525876:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346155264:0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826294.525897:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a005 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.525910:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a005 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826294.525946:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x191:0x0] 00010000:00010000:2.0:1787826294.525970:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.525985:0:6540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.525995:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.526030:0:6540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.526047:0:6540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 10 pid: 17813 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826294.526063:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346155392:192.168.203.56@tcp:104: 00000001:00080000:3.0:1787826294.529459:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973841 is committed 00000100:00100000:2.0:1787826294.541068:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826294.541475:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332555648:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 30025us (30539us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.541496:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30799 00000100:00100000:1.0:1787826294.542338:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826294.542413:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332556032 00000100:00100000:0.0:1787826294.542427:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332556032 00000100:00100000:0.0:1787826294.542498:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30801 00000100:00100000:0.0:1787826294.542839:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224234:x1874663332556032:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826294.542865:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826294.542892:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011c4 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.542934:0:8759:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.542946:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x51e:0x0].0x0 since=4s v=(0 0) 00010000:00010000:0.0:1787826294.542951:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826294.542958:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.543002:0:8759:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.543018:0:8759:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 5 pid: 8759 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826294.543048:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8759:1874663346155520:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826294.543220:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826294.544866:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.544923:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346155392:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826294.544956:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332556160 00000100:00100000:0.0:1787826294.544966:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332556160 00010000:00010000:2.0:1787826294.544968:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.545013:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 11 pid: 17813 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826294.545072:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21889 00000100:00100000:0.0:1787826294.545085:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332556160:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826294.545103:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826294.545140:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 11 pid: 17813 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.545153:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 11 pid: 17813 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.545180:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f49f4f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080118c expref: 11 pid: 17813 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.545199:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f784abfeb00) 00010000:00010000:0.0:1787826294.545204:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.545216:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.545582:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.545619:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826294.545674:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.545685:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826294.545691:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332556160:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 604us (834us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826294.545709:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21889 00010000:00010000:2.0:1787826294.545726:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.545739:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.545749:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826294.545854:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346155648:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.545978:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8759:1874663346155520:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826294.546004:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.546039:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:3.0:1787826294.550328:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826294.550603:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224234:x1874663332556032:12345-192.168.203.56@tcp:101:cat.0 Request processed in 8089us (8266us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826294.550616:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30801 00000100:00100000:1.0:1787826294.551002:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332556288 00000100:00100000:1.0:1787826294.551015:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332556288 00010000:00010000:1.0:1787826294.551066:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 6 pid: 8759 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826294.553608:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21890 00000100:00100000:1.0:1787826294.553640:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332556288:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826294.553666:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826294.553701:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 6 pid: 8759 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.553723:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 6 pid: 8759 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.553757:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008c5bf838/0xe21309aa81f49eca lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080115b expref: 6 pid: 8759 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.553790:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826294.553795:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826294.553804:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.553817:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826294.553872:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826294.553889:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6512:1874663346155776:192.168.203.56@tcp:105: 00000100:00100000:1.0:1787826294.554083:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aac00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.554313:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.554357:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826294.555133:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346155648 00000100:00100000:1.0:1787826294.555147:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346155648 00000100:00100000:1.0:1787826294.555207:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23451 00000100:00100000:1.0:1787826294.555218:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346155648:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.555344:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346155648:12345-0@lo:1000:mdt_io00_000.0 Request processed in 126us (1031us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.555352:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23451 00000100:00100000:2.0:1787826294.555484:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346155648:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.555576:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346155904:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.555611:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.555642:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826294.555792:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346155904 00000100:00100000:1.0:1787826294.555798:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346155904 00000100:00100000:1.0:1787826294.555835:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23452 00000100:00100000:1.0:1787826294.555844:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346155904:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.557137:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346155904:12345-0@lo:1000:mdt_io00_000.0 Request processed in 1286us (1516us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.557150:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23452 00000100:00100000:0.0:1787826294.558267:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346155904:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.558385:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156032:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.558459:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.558495:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ff00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826294.559049:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6512:1874663346155776:192.168.203.56@tcp:105: 00010000:00010000:1.0:1787826294.559074:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826294.559091:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826294.559149:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332556288:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 5526us (8835us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.559161:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21890 00000100:00100000:1.0:1787826294.559210:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346156032 00000100:00100000:1.0:1787826294.559218:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346156032 00000100:00100000:1.0:1787826294.559265:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23453 00000100:00100000:1.0:1787826294.559275:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346156032:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.559576:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346156032:12345-0@lo:1000:mdt_io00_000.0 Request processed in 293us (1108us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826294.559583:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23453 00000100:00100000:1.0:1787826294.560695:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156032:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.561017:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156160:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.561195:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.561230:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826294.564805:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346156160 00000100:00100000:3.0:1787826294.564827:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346156160 00000100:00100000:3.0:1787826294.564928:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23454 00000100:00100000:3.0:1787826294.564943:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346156160:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826294.565167:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346156160:12345-0@lo:1000:mdt_io00_000.0 Request processed in 227us (3970us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.565177:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23454 00000100:00100000:2.0:1787826294.565213:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156160:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.565359:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156288:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.565419:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.565438:0:24157:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332556416 00000100:00100000:0.0:1787826294.565445:0:24157:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332556416 00000100:00100000:1.0:1787826294.565445:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346156288 00000100:00100000:1.0:1787826294.565449:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346156288 00000100:00100000:1.0:1787826294.565483:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23455 00002000:00100000:0.0:1787826294.565492:0:24157:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x51e:0x0] (0->4095) req@ffff9f784f685180 x1874663332556416/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:650/0 lens 488/0 e 0 to 0 dl 1787826305 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826294.565503:0:24157:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f685180 with x1874663332556416 ext(0->4095) 00000100:00100000:2.0:1787826294.565534:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826294.565603:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346156288:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.565691:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346156288:12345-0@lo:1000:mdt_io00_000.0 Request processed in 200us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.565699:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23455 00002000:00010000:0.0:1787826294.565720:0:24157:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784f685180 00000100:00100000:0.0:1787826294.565750:0:24157:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27638 00000100:00100000:0.0:1787826294.565760:0:24157:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332556416:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:0.0:1787826294.565960:0:24157:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332556416:12345-192.168.203.56@tcp:3:cat.0 Request processed in 202us (6871us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787826294.565968:0:24157:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27638 00002000:00100000:0.0:1787826294.565975:0:24157:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x51e:0x0] (0->4095) req@ffff9f784f685180 x1874663332556416/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:650/0 lens 488/4536 e 0 to 0 dl 1787826305 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826294.565990:0:24157:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f685180 with x1874663332556416 ext(0->4095) 00002000:00010000:0.0:1787826294.565994:0:24157:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784f685180 00000100:00100000:2.0:1787826294.567944:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156288:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826294.568003:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e46:0x0] 00010000:00010000:2.0:1787826294.568196:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.568217:0:6540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.568229:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.568276:0:6540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.570009:0:6540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 9 pid: 6527 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826294.570042:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346156416:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826294.570202:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826294.571856:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.572345:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332556672 00000100:00100000:2.0:1787826294.572358:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332556672 00000100:00100000:2.0:1787826294.572427:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9319 00000100:00100000:2.0:1787826294.572439:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224234:x1874663332556672:12345-192.168.203.56@tcp:35:cat.0 00000100:00100000:2.0:1787826294.573185:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224234:x1874663332556672:12345-192.168.203.56@tcp:35:cat.0 Request processed in 744us (1336us total) trans 8589957348 rc 0/0 00000100:00100000:2.0:1787826294.573195:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9319 00000100:00100000:3.0:1787826294.578294:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826294.578462:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346156416:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826294.578508:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826294.584483:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332556800 00000100:00100000:1.0:1787826294.584507:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332556800 00010000:00010000:0.0:1787826294.584665:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 11 pid: 6527 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826294.584721:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21891 00000100:00100000:0.0:1787826294.584734:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332556800:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826294.584756:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826294.584775:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 11 pid: 6527 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.584793:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 11 pid: 6527 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.584821:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f49fe2 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801193 expref: 11 pid: 6527 timeout: 8206 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.584846:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db7:0x0].0x0 (ffff9f784abfef00) 00010000:00010000:0.0:1787826294.584852:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.584863:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.584872:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.584913:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826294.585000:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332556800:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 268us (6730us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826294.585009:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21891 00010000:00010000:2.0:1787826294.585083:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.585094:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.585212:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.585221:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.585232:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826294.585255:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156544:0@lo:101:mdt_io00_000.0 00000100:00100000:2.0:1787826294.585351:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.585408:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346156544 00000100:00100000:2.0:1787826294.585414:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346156544 00000100:00100000:2.0:1787826294.585456:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23456 00000100:00100000:2.0:1787826294.585464:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346156544:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826294.585486:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826294.585505:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4a02f lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a028 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.585529:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4a02f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a028 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.585543:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4a02f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a028 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.585552:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f784abfea00) 00010000:00010000:2.0:1787826294.585555:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787826294.585559:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826294.585592:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346156544:12345-0@lo:101:mdt_io00_000.0 Request processed in 128us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.585599:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23456 00000100:00100000:2.0:1787826294.585627:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.585656:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156544:0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826294.585672:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a02f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.585683:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a02f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826294.585706:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e47:0x0] 00000100:00100000:2.0:1787826294.585761:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156672:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.585802:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.585817:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826294.586451:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346156672 00000100:00100000:1.0:1787826294.586459:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346156672 00000100:00100000:1.0:1787826294.586508:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23457 00000100:00100000:1.0:1787826294.586518:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346156672:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826294.586688:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346156672:12345-0@lo:1000:mdt_io00_000.0 Request processed in 171us (885us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.586695:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23457 00000100:00100000:3.0:1787826294.586984:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156672:0@lo:1000:mdt_io00_000.0 00000004:00080000:3.0:1787826294.587028:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x191:0x0] 00000100:00100000:3.0:1787826294.587079:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156800:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826294.587249:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826294.587316:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.587358:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346156800 00000100:00100000:2.0:1787826294.587364:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346156800 00000100:00100000:2.0:1787826294.587436:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23458 00000100:00100000:2.0:1787826294.587446:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346156800:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.587546:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346156800:12345-0@lo:1000:mdt_io00_000.0 Request processed in 101us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.587553:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23458 00000100:00100000:0.0:1787826294.587611:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156800:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826294.587638:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e46:0x0] 00010000:00010000:0.0:1787826294.587683:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.587699:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.587710:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826294.587775:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156928:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.587812:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.589121:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826294.589248:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346156928 00000100:00100000:3.0:1787826294.589259:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346156928 00000100:00100000:3.0:1787826294.589321:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23459 00000100:00100000:3.0:1787826294.589333:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346156928:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826294.589452:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346156928:12345-0@lo:1000:mdt_io00_000.0 Request processed in 120us (1638us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.589460:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23459 00000100:00100000:0.0:1787826294.589795:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346156928:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826294.589835:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e46:0x0] 00000100:00100000:0.0:1787826294.589889:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157056:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.589925:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.589984:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826294.590016:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346157056 00000100:00100000:1.0:1787826294.590022:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346157056 00000100:00100000:3.0:1787826294.590065:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23460 00000100:00100000:3.0:1787826294.590075:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157056:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826294.590430:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157056:12345-0@lo:1000:mdt_io00_000.0 Request processed in 355us (502us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.590438:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23460 00000100:00100000:0.0:1787826294.590686:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157056:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.590737:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157184:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.590824:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.590889:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.590928:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346157184 00000100:00100000:2.0:1787826294.590934:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346157184 00000100:00100000:2.0:1787826294.591010:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23461 00000100:00100000:2.0:1787826294.591020:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157184:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.591290:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157184:12345-0@lo:1000:mdt_io00_000.0 Request processed in 270us (464us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.591298:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23461 00000100:00100000:0.0:1787826294.591395:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157184:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826294.591424:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x191:0x0] 00000100:00100000:0.0:1787826294.591461:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157312:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.591491:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.591645:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.591708:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346157312 00000100:00100000:2.0:1787826294.591714:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346157312 00000100:00100000:2.0:1787826294.591752:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23462 00000100:00100000:2.0:1787826294.591761:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851836a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157312:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.591853:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851836a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157312:12345-0@lo:1000:mdt_io00_000.0 Request processed in 92us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.591860:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23462 00000100:00100000:0.0:1787826294.591978:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157312:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826294.592001:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e46:0x0] 00000100:00100000:0.0:1787826294.592050:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157440:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.592077:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.592100:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826294.592242:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346157440 00000100:00100000:3.0:1787826294.592248:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346157440 00000100:00100000:3.0:1787826294.592276:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23463 00000100:00100000:3.0:1787826294.592284:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157440:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826294.592355:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157440:12345-0@lo:1000:mdt_io00_000.0 Request processed in 71us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.592362:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23463 00000100:00100000:0.0:1787826294.592467:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157440:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.592501:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157568:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.592695:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.592760:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826294.592822:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346157568 00000100:00100000:3.0:1787826294.592827:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346157568 00000100:00100000:3.0:1787826294.592865:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23464 00000100:00100000:3.0:1787826294.592874:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157568:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826294.592935:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157568:12345-0@lo:1000:mdt_io00_000.0 Request processed in 62us (408us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.592942:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23464 00000100:00100000:0.0:1787826294.593008:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157568:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.593098:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157696:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826294.593169:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.593183:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.593467:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346157696 00000100:00100000:2.0:1787826294.593473:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346157696 00000100:00100000:2.0:1787826294.593509:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23465 00000100:00100000:2.0:1787826294.593517:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157696:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826294.593777:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346157696:12345-0@lo:1000:mdt_io00_000.0 Request processed in 260us (607us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.593785:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23465 00000100:00100000:0.0:1787826294.593861:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346157696:0@lo:1000:mdt_io00_000.0 00000040:00080000:0.0:1787826294.594819:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].211, 64 off21696 00000004:00080000:0.0:1787826294.594835:0:6540:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8106806728566ns, next at 8107806725161ns, rc = 0 00000040:00080000:0.0:1787826294.594901:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].932, 904 off876960 00000040:00080000:0.0:1787826294.594958:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].935, 984 off975864 00000004:00080000:0.0:1787826294.594966:0:6540:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784549f540 with version 1818 00000004:00080000:3.0:1787826294.595219:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1818 rpc_version 1818 00000020:00100000:3.0:1787826294.595264:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826294.595286:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784a91d180 x1874663346157952/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2584/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826294.595314:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d180 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346157952:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826294.595391:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826294.595431:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826294.595655:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346157952 00000100:00100000:2.0:1787826294.595665:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346157952 00000100:00100000:2.0:1787826294.595734:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23466 00000100:00100000:2.0:1787826294.595745:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf88850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346157952:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826294.596199:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf88850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346157952:12345-0@lo:1000:osp_up1-0.0 Request processed in 446us (798us total) trans 4294985359 rc 0/0 00000100:00100000:2.0:1787826294.596209:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23466 00000100:00100000:3.0:1787826294.596746:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d180 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346157952:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787826294.596851:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596869:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596878:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596884:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826294.596889:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596897:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596913:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db7:0x0].0x0 (ffff9f784abfef00) 00010000:00010000:0.0:1787826294.596916:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826294.596922:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596928:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a021 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826294.596954:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e47:0x0] 00010000:00010000:0.0:1787826294.596962:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a02f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596971:0:6540:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a02f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596986:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596992:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.596998:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597002:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826294.597005:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597011:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597019:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f784abfeb00) 00010000:00010000:0.0:1787826294.597022:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826294.597024:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597030:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a00c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826294.597043:0:6540:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c43a9c6e reply state 000000004ef74379 transno 8589957349 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597055:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597063:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f7952ca9000) 00010000:00010000:0.0:1787826294.597066:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826294.597069:0:6540:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:0.0:1787826294.597073:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x191:0x0] 00010000:00010000:0.0:1787826294.597077:0:6540:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597084:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597092:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597155:0:6540:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826294.597161:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x191:0x0] 00010000:00010000:0.0:1787826294.597175:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a005 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597183:0:6540:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a005 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597191:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597196:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597202:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597207:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826294.597220:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597226:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597235:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f784b4a5300) 00010000:00010000:0.0:1787826294.597237:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826294.597240:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597246:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a036 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597279:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597290:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597298:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597306:0:6540:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597313:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597318:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826294.597344:0:6540:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1787826294.597379:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826294.597390:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597403:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826294.597410:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826294.597420:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597464:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 2/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597471:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826294.597507:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 1/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597515:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43000/0xe21309aa81f49fdb lrc: 0/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f49ff0 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597536:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826294.597540:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597547:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826294.597551:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597563:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597576:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db7:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:3.0:1787826294.597579:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787826294.597583:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597589:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a03d lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826294.597603:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826294.597604:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826294.597606:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826294.597616:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826294.597633:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346158080:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826294.597691:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826294.597903:0:6540:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224236:x1874663332555136:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 109727us (112111us total) trans 8589957349 rc 0/0 00000100:00100000:0.0:1787826294.597914:0:6540:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1226 00010000:00010000:2.0:1787826294.598945:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826294.598972:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346158080 00000100:00100000:2.0:1787826294.598985:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346158080 00000100:00100000:2.0:1787826294.599170:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11039 00000100:00100000:2.0:1787826294.599201:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+9:3652:x1874663346158080:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826294.599221:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826294.599250:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f49ff0 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49fdb expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826294.599272:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:2.0:1787826294.599283:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826294.599358:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346158080:12345-0@lo:103:ldlm_bl.0 Request processed in 167us (1665us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.599365:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11039 00000100:00100000:1.0:1787826294.599456:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346158080:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826294.770350:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346158208:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.770477:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.770549:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346158720:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.770584:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.770606:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346159232:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.770637:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.770666:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1787826294.770672:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346158336:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.770702:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.770723:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346158848:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.770761:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.770788:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346159360:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.770812:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826294.770853:0:30904:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346158208 00000100:00100000:2.0:1787826294.770863:0:30904:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346158208 00000100:00100000:2.0:1787826294.770921:0:30904:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2215 00000100:00100000:2.0:1787826294.770933:0:30904:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf43c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346158208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.770998:0:30904:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf43c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346158208:12345-0@lo:400:kworker.0 Request processed in 67us (521us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.771006:0:30904:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2215 00000100:00100000:2.0:1787826294.771054:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346159232 00000100:00100000:2.0:1787826294.771058:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346159232 00000100:00100000:2.0:1787826294.771083:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346158848 00000100:00100000:2.0:1787826294.771086:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346158848 00000100:00100000:2.0:1787826294.771143:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346159360 00000100:00100000:2.0:1787826294.771146:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346159360 00000100:00100000:2.0:1787826294.771168:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8786 00000100:00100000:2.0:1787826294.771176:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346159232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.771212:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346159232:12345-0@lo:400:kworker.0 Request processed in 37us (574us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.771219:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8786 00000100:00100000:2.0:1787826294.771234:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8787 00000100:00100000:2.0:1787826294.771240:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346158848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.771268:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346158848:12345-0@lo:400:kworker.0 Request processed in 29us (507us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.771274:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8787 00000100:00100000:2.0:1787826294.771284:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8788 00000100:00100000:2.0:1787826294.771290:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346159360:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.771314:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346159360:12345-0@lo:400:kworker.0 Request processed in 24us (501us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.771321:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8788 00000100:00080000:2.0:1787826294.771338:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826294.771369:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973838 gen 2 00000004:00080000:2.0:1787826294.771377:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784c6bb480, transno 4294973838 00000100:00100000:2.0:1787826294.771390:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346158848:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.771401:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973840 gen 2 00000004:00080000:2.0:1787826294.771405:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7849dd0380, transno 4294973840 00000100:00100000:2.0:1787826294.771413:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346159360:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.771424:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346158208:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.771432:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976149 gen 2 00000004:00080000:2.0:1787826294.771435:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7850ad6a00, transno 4294976149 00000100:00100000:2.0:1787826294.771444:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346159232:0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.771664:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346158336 00000100:00100000:2.0:1787826294.771669:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346158336 00000100:00100000:2.0:1787826294.771701:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23467 00000100:00100000:2.0:1787826294.771708:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346158336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826294.771759:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346158336:12345-0@lo:400:kworker.0 Request processed in 51us (1056us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826294.771781:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23467 00000100:00100000:2.0:1787826294.771810:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346158336:0@lo:400:kworker.0 00000100:00100000:3.0:1787826294.772042:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346158720 00000100:00100000:3.0:1787826294.772047:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346158720 00000100:00100000:3.0:1787826294.772094:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14258 00000100:00100000:3.0:1787826294.772138:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346158720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826294.772189:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346158720:12345-0@lo:400:kworker.0 Request processed in 88us (1604us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826294.772196:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14258 00000100:00080000:3.0:1787826294.772219:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826294.772239:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346158720:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.772862:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346158592:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.772927:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.772971:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346159104:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.772995:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.773015:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346159616:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773037:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.773059:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1787826294.773066:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346158464:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773091:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.773136:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346158976:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773160:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.773182:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346159488:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773208:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826294.773248:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346159488 00000100:00100000:1.0:1787826294.773255:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346159488 00000100:00100000:1.0:1787826294.773301:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8789 00000100:00100000:1.0:1787826294.773310:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346159488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773356:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346159488:12345-0@lo:400:kworker.0 Request processed in 48us (147us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.773366:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8789 00000100:00080000:1.0:1787826294.773392:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826294.773403:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346158592 00000100:00100000:1.0:1787826294.773406:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346158592 00000100:00100000:1.0:1787826294.773437:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346159104 00000100:00100000:1.0:1787826294.773440:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346159104 00000100:00100000:1.0:1787826294.773462:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346159616 00000100:00100000:1.0:1787826294.773464:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346159616 00000100:00100000:1.0:1787826294.773484:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346158464 00000100:00100000:1.0:1787826294.773493:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346158464 00000100:00100000:1.0:1787826294.773516:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346158976 00000100:00100000:1.0:1787826294.773519:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346158976 00000100:00100000:1.0:1787826294.773541:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14259 00000100:00100000:1.0:1787826294.773547:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346158592:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773582:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346158592:12345-0@lo:400:kworker.0 Request processed in 34us (654us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.773588:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14259 00000100:00100000:1.0:1787826294.773602:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14260 00000100:00100000:1.0:1787826294.773608:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346159104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773635:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346159104:12345-0@lo:400:kworker.0 Request processed in 28us (639us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.773640:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14260 00000100:00100000:1.0:1787826294.773655:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14261 00000100:00100000:1.0:1787826294.773661:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346159616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773686:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346159616:12345-0@lo:400:kworker.0 Request processed in 26us (648us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.773691:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14261 00000100:00100000:1.0:1787826294.773703:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14262 00000100:00100000:1.0:1787826294.773716:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346158464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773744:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346158464:12345-0@lo:400:kworker.0 Request processed in 35us (652us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.773750:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14262 00000100:00100000:1.0:1787826294.773761:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14263 00000100:00100000:1.0:1787826294.773779:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346158976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773809:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346158976:12345-0@lo:400:kworker.0 Request processed in 31us (648us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826294.773816:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14263 00000100:00080000:1.0:1787826294.773850:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826294.773853:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826294.773895:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346158464:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773907:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346158976:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773917:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346159488:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773927:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346158592:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773935:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346159104:0@lo:400:kworker.0 00000100:00100000:1.0:1787826294.773944:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346159616:0@lo:400:kworker.0 00010000:00010000:1.0:1787826294.962426:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826295.276619:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787826295.276645:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346159744:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826295.276715:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826295.276842:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346159744 00000100:00100000:2.0:1787826295.276850:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346159744 00000100:00100000:2.0:1787826295.276921:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6211 00000100:00100000:2.0:1787826295.276931:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346159744:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1787826295.276954:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683135488 00000100:00100000:2.0:1787826295.277007:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346159744:12345-0@lo:13:osp-pre-1-1.0 Request processed in 77us (290us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.277014:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6211 00000100:00100000:2.0:1787826295.277071:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346159744:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1787826295.277170:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787826295.410170:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a005 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826295.410191:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a005 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.410237:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a005 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.410244:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826295.410268:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efc00/0xe21309aa81f49ffe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a005 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.410290:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a02f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826295.410297:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a02f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.410313:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a02f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.410318:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826295.410333:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490efa00/0xe21309aa81f4a028 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a02f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826295.410352:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957349 is committed 00000004:00080000:2.0:1787826295.410360:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 216 committed 00000001:00080000:2.0:1787826295.410375:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957348 is committed 00000020:00080000:2.0:1787826295.410384:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979078 batchid updated 00000001:00080000:2.0:1787826295.410394:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957346 is committed 00010000:00010000:1.0:1787826295.410395:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.410403:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.410414:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.410416:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.410423:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.410428:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.418294:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346160000:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826295.418367:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.418414:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826295.418422:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346159872:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826295.418442:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.418471:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346160000 00000100:00100000:3.0:1787826295.418478:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346160000 00000100:00100000:3.0:1787826295.418520:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346159872 00000100:00100000:3.0:1787826295.418522:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346159872 00000100:00100000:3.0:1787826295.418547:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11040 00000100:00100000:3.0:1787826295.418555:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3652:x1874663346160000:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826295.418569:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826295.418594:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4a02f lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a028 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.418602:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826295.418603:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826295.418613:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f784abfea00) 00010000:00010000:2.0:1787826295.418614:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.418622:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.418624:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826295.418632:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f7952ca9000) 00010000:00010000:2.0:1787826295.418635:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826295.418638:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.418645:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a01a lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.418656:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346160000:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 102us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826295.418663:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11040 00010000:00010000:1.0:1787826295.418676:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.418683:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11041 00010000:00010000:1.0:1787826295.418693:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.418693:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346159872:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826295.418707:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826295.418712:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f7952ca9000) 00010000:00010000:1.0:1787826295.418716:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826295.418720:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.418721:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4a005 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f49ffe expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.418739:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f49eb5 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787826295.418739:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979078 00010000:00010000:3.0:1787826295.418743:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x4ec94fd6 (ffff9f784abfed00) 00010000:00010000:3.0:1787826295.418754:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826295.418796:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346159872:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 105us (354us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826295.418802:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11041 00000004:00080000:2.0:1787826295.418805:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7851057b40 with version 1819 00000020:00080000:2.0:1787826295.418829:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979078 cancel update log [0x240000404:0x2:0x0].934: rc = 0 00000100:00100000:3.0:1787826295.418847:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346159872:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826295.418874:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346160000:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826295.418883:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826295.418888:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346160128:0@lo:6:osp-syn-1-0.0 00000020:00080000:2.0:1787826295.418902:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979078 cancel update log [0x200000bd0:0x2:0x0].931: rc = 0 00000020:00080000:2.0:1787826295.418910:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979078 00000100:00100000:3.0:1787826295.418914:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.418942:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346160128 00000100:00100000:3.0:1787826295.418947:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346160128 00000100:00100000:3.0:1787826295.418986:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30802 00000100:00100000:3.0:1787826295.418993:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346160128:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1787826295.419015:0:17826:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1310 count 1 00010000:00010000:3.0:1787826295.419035:0:17826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.419055:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.419064:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=1s v=(1 0) 00010000:00010000:3.0:1787826295.419068:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826295.419073:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.419100:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.419193:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 5 pid: 8759 timeout: 8207 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.419203:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346160256:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826295.419260:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bc00 going to sleep for 11 seconds 00000004:00080000:1.0:1787826295.421341:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1819 rpc_version 1819 00000020:00100000:1.0:1787826295.421379:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826295.421390:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7848303b80 x1874663346160512/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826295.421414:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346160512:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826295.421466:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.421504:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aac00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826295.421554:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346160512 00000100:00100000:3.0:1787826295.421561:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346160512 00000100:00100000:2.0:1787826295.421571:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346160256:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826295.421596:0:17826:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.421615:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23468 00000100:00100000:3.0:1787826295.421625:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346160512:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826295.421795:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346160512:12345-0@lo:1000:osp_up1-0.0 Request processed in 171us (327us total) trans 4294985360 rc 0/0 00000100:00100000:3.0:1787826295.421803:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23468 00000100:00100000:2.0:1787826295.422273:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346160512:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826295.433182:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.433381:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332556928 00000100:00100000:2.0:1787826295.433391:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332556928 00010000:00010000:2.0:1787826295.433453:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 6 pid: 8759 timeout: 8207 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.433498:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21892 00000100:00100000:2.0:1787826295.433508:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd83050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332556928:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826295.433523:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826295.433529:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 6 pid: 8759 timeout: 8207 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433542:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 6 pid: 8759 timeout: 8207 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433564:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4a013 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608011c4 expref: 6 pid: 8759 timeout: 8207 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433586:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826295.433591:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826295.433597:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433608:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433638:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826295.433698:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd83050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332556928:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 192us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.433706:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21892 00010000:00010000:2.0:1787826295.433772:0:17826:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433782:0:17826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433793:0:17826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433800:0:17826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.433816:0:17826:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826295.433986:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346160128:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14993us (15069us total) trans 4294976151 rc 0/0 00000100:00100000:2.0:1787826295.433994:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30802 00000100:00080000:1.0:1787826295.434685:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787826295.435260:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826295.435280:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.435293:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826295.435298:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.435308:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.435325:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.435332:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000acfa8029/0xe21309aa81f4a044 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.435354:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826295.435356:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826295.435357:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1787826295.435504:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f78435b3480/2, rc 0, transno 4294976151 00000100:00100000:3.0:1787826295.435513:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346160128:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787826295.783188:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826295.783273:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332557056 00000100:00100000:1.0:1787826295.783283:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332557056 00000100:00100000:1.0:1787826295.783356:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68316 00000100:00100000:1.0:1787826295.783367:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224272:x1874663332557056:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826295.783387:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826295.783411:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a04b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011cb expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.783484:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a052 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.783499:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a052 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.783510:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a052 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826295.783604:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e46:0x0]: returning size 4096 00000004:00200000:1.0:1787826295.783609:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787826295.783628:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a052 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.783654:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a04b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011cb expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.783675:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a052 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011cb expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.783685:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1800) 00010000:00010000:1.0:1787826295.783687:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826295.783691:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826295.783748:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224272:x1874663332557056:12345-192.168.203.56@tcp:101:rm.0 Request processed in 383us (569us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.783768:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68316 00000100:00080000:0.0:1787826295.784185:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826295.787940:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826295.788012:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332557184 00000100:00100000:1.0:1787826295.788022:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332557184 00000100:00100000:1.0:1787826295.788088:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68317 00000100:00100000:1.0:1787826295.788099:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224272:x1874663332557184:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826295.788174:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826295.788196:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a059 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011d2 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826295.788359:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346160640:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826295.788405:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826295.788437:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826295.788443:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346160768:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826295.788469:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826295.788503:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346160768 00000100:00100000:0.0:1787826295.788507:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346160768 00000100:00100000:0.0:1787826295.788540:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23469 00000100:00100000:0.0:1787826295.788546:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346160768:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1787826295.788574:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443422208 avail 1297182720 00000100:00100000:0.0:1787826295.788628:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346160768:12345-0@lo:41:osp-pre-1-0.0 Request processed in 81us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826295.788635:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23469 00000100:00100000:0.0:1787826295.788704:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346160768:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1787826295.788905:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.788918:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.788928:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826295.788966:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x191:0x0]: returning size 4096 00000004:00200000:1.0:1787826295.788970:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826295.788998:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a059 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.789013:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.789021:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7967ac1100) 00010000:00010000:1.0:1787826295.789025:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826295.789028:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826295.789072:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224272:x1874663332557184:12345-192.168.203.56@tcp:101:rm.0 Request processed in 975us (1141us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.789078:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68317 00000100:00100000:2.0:1787826295.789383:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346160640 00000100:00100000:2.0:1787826295.789389:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346160640 00000100:00100000:2.0:1787826295.789417:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6212 00000100:00100000:2.0:1787826295.789424:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851836a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346160640:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1787826295.789437:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662430208 00000100:00100000:2.0:1787826295.789474:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851836a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346160640:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (1068us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.789480:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6212 00000100:00100000:2.0:1787826295.789541:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346160896:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826295.789570:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826295.789603:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826295.789609:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346161024:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826295.789631:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826295.789671:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346160896 00000100:00100000:2.0:1787826295.789675:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346160896 00000100:00100000:2.0:1787826295.789698:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346161024 00000100:00100000:2.0:1787826295.789700:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346161024 00000100:00100000:2.0:1787826295.789713:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6213 00000100:00100000:2.0:1787826295.789718:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346160896:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826295.789729:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683135488 00000100:00100000:2.0:1787826295.789770:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346160896:12345-0@lo:13:osp-pre-1-0.0 Request processed in 51us (199us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.789776:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6213 00000100:00100000:2.0:1787826295.789788:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6214 00000100:00100000:2.0:1787826295.789794:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346161024:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826295.789802:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662430208 00000100:00100000:2.0:1787826295.789822:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346161024:12345-0@lo:13:osp-pre-0-0.0 Request processed in 29us (190us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.789827:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6214 00000100:00100000:2.0:1787826295.789858:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346161024:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826295.789868:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346160896:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826295.790523:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346160640:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826295.792561:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826295.792585:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346161152:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826295.792672:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.798320:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346161152 00000100:00100000:3.0:1787826295.798335:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346161152 00000100:00100000:1.0:1787826295.798358:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826295.798387:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23470 00000100:00100000:3.0:1787826295.798396:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346161152:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787826295.798432:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442811904 avail 1296572416 00000100:00100000:3.0:1787826295.798518:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346161152:12345-0@lo:41:osp-pre-0-1.0 Request processed in 122us (5845us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826295.798526:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23470 00000100:00100000:1.0:1787826295.798580:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957346 gen 2 00000004:00080000:1.0:1787826295.798595:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957343 committed transno 8589957346 00000004:00080000:1.0:1787826295.798615:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957344 committed transno 8589957346 00000004:00080000:1.0:1787826295.798625:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957345 committed transno 8589957346 00000004:00080000:1.0:1787826295.798633:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957346 committed transno 8589957346 00000100:00100000:1.0:1787826295.798718:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346161152:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826295.798921:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332557312 00000100:00100000:2.0:1787826295.798926:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332557312 00000100:00100000:2.0:1787826295.798987:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68318 00000100:00100000:2.0:1787826295.798996:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224272:x1874663332557312:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826295.799010:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826295.799042:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a067 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011d9 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.799100:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.799135:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.799148:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826295.799198:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x18f:0x0]: returning size 4096 00000004:00200000:2.0:1787826295.799213:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826295.799248:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a067 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.799265:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.799274:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:2.0:1787826295.799277:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826295.799280:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826295.799323:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332557312:12345-192.168.203.56@tcp:101:rm.0 Request processed in 329us (969us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.799331:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68318 00000100:00080000:1.0:1787826295.801191:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826295.811806:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.812333:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332557568 00000100:00100000:2.0:1787826295.812343:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332557568 00000100:00100000:2.0:1787826295.812406:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68319 00000100:00100000:2.0:1787826295.812417:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224272:x1874663332557568:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826295.812438:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826295.812460:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4a075 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011e0 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826295.812620:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e46:0x0]: returning size 4096 00010000:00010000:2.0:1787826295.812713:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4a075 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011e0 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.812732:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f79462e8300) 00010000:00010000:2.0:1787826295.812735:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826295.812740:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4a075 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011e0 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.812766:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826295.812836:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332557568:12345-192.168.203.56@tcp:101:rm.0 Request processed in 421us (1039us total) trans 4294985361 rc 301/301 00000100:00100000:2.0:1787826295.812843:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68319 00000100:00100000:0.0:1787826295.818678:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.818752:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332557696 00000100:00100000:2.0:1787826295.818772:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332557696 00000100:00100000:2.0:1787826295.818830:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9320 00000100:00100000:2.0:1787826295.818840:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332557696:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826295.825744:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332557696:12345-192.168.203.56@tcp:37:rm.0 Request processed in 6894us (7067us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.825768:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9320 00000100:00100000:0.0:1787826295.837324:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826295.838571:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332557824 00000100:00100000:3.0:1787826295.838583:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332557824 00000100:00100000:3.0:1787826295.838659:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9321 00000100:00100000:3.0:1787826295.838669:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332557824:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826295.852825:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332557824:12345-192.168.203.56@tcp:37:rm.0 Request processed in 14147us (15501us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.852839:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9321 00000100:00100000:3.0:1787826295.866312:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826295.866561:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332558080 00000100:00100000:0.0:1787826295.866575:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332558080 00000100:00100000:0.0:1787826295.866709:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9322 00000100:00100000:0.0:1787826295.866723:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd3850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332558080:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:0.0:1787826295.867085:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1186, 344 off1079400 00000040:00080000:0.0:1787826295.867180:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1505, 424 off1520192 00000004:00080000:0.0:1787826295.867188:0:6531:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852405600 with version 6804 00000004:00080000:3.0:1787826295.867502:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6804 rpc_version 6804 00000020:00100000:3.0:1787826295.867542:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826295.867556:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5c9180 x1874663346161408/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826295.867589:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c9180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346161408:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826295.867668:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.867708:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826295.867990:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346161408 00000100:00100000:2.0:1787826295.868164:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346161408 00000100:00100000:2.0:1787826295.868290:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23471 00000100:00100000:2.0:1787826295.868302:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346161408:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826295.868556:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346161408:12345-0@lo:1000:osp_up0-1.0 Request processed in 256us (886us total) trans 8589957350 rc 0/0 00000100:00100000:2.0:1787826295.868565:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23471 00000100:00100000:2.0:1787826295.868659:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c9180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346161408:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826295.868807:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd3850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224272:x1874663332558080:12345-192.168.203.56@tcp:35:rm.0 Request processed in 2085us (2507us total) trans 4294985362 rc 0/0 00000100:00100000:2.0:1787826295.868817:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9322 00000100:00100000:3.0:1787826295.873938:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826295.874979:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332558336 00000100:00100000:1.0:1787826295.874997:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332558336 00000100:00100000:1.0:1787826295.875321:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68320 00000100:00100000:1.0:1787826295.875336:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224272:x1874663332558336:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826295.875383:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826295.875414:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a083 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011e7 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.875621:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a08a lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.875638:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a08a lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.875655:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a08a lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826295.875668:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e46:0x0]: returning size 4096 00010000:00010000:1.0:1787826295.875807:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a083 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011e7 expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.875836:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a08a lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011e7 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.875858:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1800) 00010000:00010000:1.0:1787826295.875861:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1787826295.875866:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826295.875950:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224272:x1874663332558336:12345-192.168.203.56@tcp:101:rm.0 Request processed in 617us (2027us total) trans 4294985363 rc 0/0 00000100:00100000:1.0:1787826295.875959:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68320 00000100:00100000:1.0:1787826295.880588:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.880709:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332558464 00000100:00100000:2.0:1787826295.880725:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332558464 00000100:00100000:2.0:1787826295.880854:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68321 00000100:00100000:2.0:1787826295.880869:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332558464:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:2.0:1787826295.880941:0:35080:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:2.0:1787826295.881036:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332558464:12345-192.168.203.56@tcp:41:rm.0 Request processed in 172us (463us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.881045:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68321 00000100:00100000:1.0:1787826295.885594:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.885690:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332558592 00000100:00100000:2.0:1787826295.885704:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332558592 00000100:00100000:2.0:1787826295.885785:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68322 00000100:00100000:2.0:1787826295.885801:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224272:x1874663332558592:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826295.885832:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826295.885868:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4a098 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011ee expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.885956:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.885975:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.885988:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886010:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886018:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886025:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886134:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a0ad lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886143:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a0ad lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886152:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a0ad lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826295.886178:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a0ad lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886195:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886203:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886211:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886479:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886487:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826295.886491:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886501:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886517:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:2.0:1787826295.886520:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826295.886526:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 3/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886533:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a09f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886546:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886551:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886558:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886563:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826295.886566:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886572:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886583:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f79462e8300) 00010000:00010000:2.0:1787826295.886586:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826295.886589:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886595:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886638:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4a098 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011ee expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886655:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a0ad lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011ee expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.886662:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f79462e8400) 00010000:00010000:2.0:1787826295.886665:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826295.886668:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826295.886796:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224272:x1874663332558592:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1000us (1217us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.886806:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68322 00000100:00100000:1.0:1787826295.891417:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.891518:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332558720 00000100:00100000:2.0:1787826295.891531:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332558720 00000100:00100000:2.0:1787826295.891631:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68323 00000100:00100000:2.0:1787826295.891644:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224272:x1874663332558720:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826295.891676:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826295.891706:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4a0b4 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011f5 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.891800:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4a0bb lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.891817:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4a0bb lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.891830:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4a0bb lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826295.891966:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e47:0x0]: returning size 6 00000004:00200000:2.0:1787826295.891972:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:2.0:1787826295.892032:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4a0b4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011f5 expref: 14 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.892064:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4a0bb lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011f5 expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.892073:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f7967ac8f00) 00010000:00010000:2.0:1787826295.892077:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826295.892080:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826295.892191:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224272:x1874663332558720:12345-192.168.203.56@tcp:101:rm.0 Request processed in 549us (788us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.892200:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68323 00000100:00100000:3.0:1787826295.896692:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826295.896749:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826295.896804:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332558848 00000100:00100000:3.0:1787826295.896804:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332559104 00000100:00100000:1.0:1787826295.896819:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332558848 00000100:00100000:3.0:1787826295.896819:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332559104 00000100:00100000:3.0:1787826295.896903:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68324 00000100:00100000:1.0:1787826295.896904:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11042 00000100:00100000:1.0:1787826295.896922:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332558848:12345-192.168.203.56@tcp:103:rm.0 00000100:00100000:3.0:1787826295.896924:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224272:x1874663332559104:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826295.896959:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826295.896993:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a0ad lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011ee expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.896993:0:24703:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826295.897017:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4a0bb lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e47:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011f5 expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.897021:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f79462e8400) 00010000:00010000:1.0:1787826295.897035:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826295.897046:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f7967ac8f00) 00010000:00010000:3.0:1787826295.897065:0:24703:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826295.897132:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332558848:12345-192.168.203.56@tcp:103:rm.0 Request processed in 187us (423us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.897142:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11042 00010000:00010000:3.0:1787826295.897145:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.897155:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.897168:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.897197:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346161536:0@lo:101:mdt00_007.0 00000100:00100000:3.0:1787826295.897493:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.897537:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826295.897599:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346161536 00000100:00100000:1.0:1787826295.897604:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346161536 00000100:00100000:1.0:1787826295.897648:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23472 00000100:00100000:1.0:1787826295.897656:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346161536:12345-0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826295.897673:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826295.897687:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a0c2 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.897712:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.897722:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a0c2 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.897748:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.897772:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 8 pid: 35098 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.897783:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a6680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346161664:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826295.897833:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826295.901794:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.901896:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332559232 00000100:00100000:2.0:1787826295.901905:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332559232 00000100:00100000:1.0:1787826295.901914:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a6680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346161664:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826295.901951:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 9 pid: 35098 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.901958:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0c2 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.901987:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:1.0:1787826295.901992:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826295.901997:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0c2 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.902009:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:2.0:1787826295.902021:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21893 00000100:00100000:2.0:1787826295.902031:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332559232:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826295.902047:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826295.902055:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 9 pid: 35098 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902070:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 9 pid: 35098 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902098:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a06e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x18f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d9 expref: 9 pid: 35098 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.902136:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:24703:x1874663346161536:12345-0@lo:101:mdt00_007.0 Request processed in 4447us (4609us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.902148:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23472 00010000:00010000:2.0:1787826295.902149:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:2.0:1787826295.902155:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0c2 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902166:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a0c2 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902176:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a0c2 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902215:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0c2 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.902229:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346161792:0@lo:105: 00000100:00100000:2.0:1787826295.902276:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826295.902316:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3b480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826295.902333:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346161792 00000100:00100000:2.0:1787826295.902336:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346161792 00000100:00100000:2.0:1787826295.902369:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7833 00000100:00100000:2.0:1787826295.902377:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346161792:12345-0@lo:105: 00010000:00010000:2.0:1787826295.902389:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902401:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902410:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902418:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902425:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902434:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.902450:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69600) 00000100:00100000:2.0:1787826295.902481:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346161792:12345-0@lo:105: Request processed in 105us (204us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.902487:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7833 00000100:00100000:1.0:1787826295.902534:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346161536:0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826295.902559:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826295.902586:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18f:0x0] 00010000:00010000:1.0:1787826295.902628:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.902636:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.902646:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.902667:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a6680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346161920:0@lo:101:mdt00_007.0 00000100:00100000:1.0:1787826295.902700:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.902722:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826295.902740:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346161920 00000100:00100000:1.0:1787826295.902744:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346161920 00000100:00100000:1.0:1787826295.902784:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23473 00000100:00100000:1.0:1787826295.902793:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:24703:x1874663346161920:12345-0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826295.902811:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826295.902826:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a0d7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a0d0 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.902846:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a0d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a0d0 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.902858:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a0d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0d0 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.902865:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f7967ac1a00) 00010000:00010000:1.0:1787826295.902868:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826295.902871:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826295.902903:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346161920:12345-0@lo:101:mdt00_007.0 Request processed in 112us (202us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.902909:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23473 00000100:00100000:1.0:1787826295.902937:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a6680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346161920:0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826295.902948:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a0d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.902962:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a0d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826295.902975:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18f:0x0] 00000100:00100000:1.0:1787826295.903013:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346162048:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826295.903063:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346161792:0@lo:105: 00010000:00010000:2.0:1787826295.903080:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826295.903148:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2396:x1874663332559232:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1114us (1363us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.903157:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21893 00000100:00100000:1.0:1787826295.904460:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.904490:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826295.904613:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346162048 00000100:00100000:3.0:1787826295.904623:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346162048 00000100:00100000:3.0:1787826295.904686:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23474 00000100:00100000:3.0:1787826295.904698:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346162048:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826295.904823:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346162048:12345-0@lo:1000:mdt00_007.0 Request processed in 127us (368us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826295.904831:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23474 00000100:00100000:2.0:1787826295.904908:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346162048:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826295.904992:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346162176:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826295.905041:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.905068:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346162176 00000100:00100000:2.0:1787826295.905072:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849c3bf00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826295.905075:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346162176 00000100:00100000:1.0:1787826295.905141:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23475 00000100:00100000:1.0:1787826295.905148:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346162176:12345-0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826295.905568:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346162176:12345-0@lo:1000:mdt00_007.0 Request processed in 419us (524us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.905576:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23475 00000100:00100000:3.0:1787826295.905618:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3800 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346162176:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826295.905682:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346162304:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826295.905716:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.905741:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826295.905814:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346162304 00000100:00100000:2.0:1787826295.905820:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346162304 00000100:00100000:2.0:1787826295.905861:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23476 00000100:00100000:2.0:1787826295.905869:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346162304:12345-0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826295.905974:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346162304:12345-0@lo:1000:mdt00_007.0 Request processed in 105us (256us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826295.905981:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23476 00000100:00100000:1.0:1787826295.906043:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346162304:0@lo:1000:mdt00_007.0 00010000:00010000:1.0:1787826295.906153:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.906176:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.906188:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.906204:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346162432:0@lo:101:mdt00_007.0 00000100:00100000:1.0:1787826295.906571:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.906598:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826295.906664:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346162432 00000100:00100000:2.0:1787826295.906671:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346162432 00000100:00100000:2.0:1787826295.906715:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23477 00000100:00100000:2.0:1787826295.906724:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346162432:12345-0@lo:101:mdt00_007.0 00010000:00010000:2.0:1787826295.906735:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826295.906765:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4a0e5 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a0de expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.906798:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4a0e5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a0de expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.906819:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4a0e5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0de expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.906830:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f7967ac8b00) 00010000:00010000:2.0:1787826295.906834:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826295.906838:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826295.906880:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:24703:x1874663346162432:12345-0@lo:101:mdt00_007.0 Request processed in 158us (310us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826295.906887:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23477 00000100:00100000:1.0:1787826295.906952:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346162432:0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826295.906969:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.906981:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.907016:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.907025:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.907033:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787826295.907728:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].225, 64 off22592 00000004:00080000:1.0:1787826295.907742:0:24703:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8108119635806ns, next at 8109119633152ns, rc = 0 00000040:00080000:1.0:1787826295.907796:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1187, 384 off1079784 00000040:00080000:1.0:1787826295.907858:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1506, 464 off1520656 00000004:00080000:1.0:1787826295.907864:0:24703:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852f25cc0 with version 6805 00000004:00080000:3.0:1787826295.908910:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6805 rpc_version 6805 00000020:00100000:3.0:1787826295.908955:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826295.908969:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b753480 x1874663346162688/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826295.908996:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346162688:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826295.909080:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.909142:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826295.909232:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346162688 00000100:00100000:2.0:1787826295.909241:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346162688 00000100:00100000:2.0:1787826295.909304:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23478 00000100:00100000:2.0:1787826295.909316:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346162688:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826295.909685:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346162688:12345-0@lo:1000:osp_up0-1.0 Request processed in 370us (604us total) trans 8589957351 rc 0/0 00000100:00100000:2.0:1787826295.909694:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23478 00000100:00100000:3.0:1787826295.910261:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b753480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346162688:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826295.910492:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910512:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910525:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910536:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910548:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910554:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910570:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787826295.910597:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910604:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910611:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910617:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826295.910622:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910630:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910644:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:2.0:1787826295.910648:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826295.910652:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910658:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a0ec lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910677:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910685:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910692:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910709:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826295.910724:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18f:0x0] 00010000:00010000:2.0:1787826295.910732:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910739:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910747:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910776:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:2.0:1787826295.910781:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18f:0x0] 00010000:00010000:2.0:1787826295.910786:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a0d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910795:0:24703:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a0d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.910884:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224272:x1874663332559104:12345-192.168.203.56@tcp:36:rm.0 Request processed in 13965us (14132us total) trans 4294985364 rc 0/0 00000100:00100000:2.0:1787826295.910893:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68324 00010000:00010000:2.0:1787826295.910925:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1787826295.910932:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910939:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826295.910943:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910950:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910959:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:2.0:1787826295.910962:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826295.910965:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910972:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a0f3 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e47:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.910990:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826295.910994:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.911000:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826295.911005:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826295.911015:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.911145:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826295.911151:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.911160:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826295.911165:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826295.911172:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.911195:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.911202:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826295.911239:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.911248:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4a0c2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a0c9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.911265:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826295.911267:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826295.911269:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826295.911451:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.911461:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826295.911480:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.911488:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4a0de lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a0e5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.911503:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826295.911505:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826295.911506:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826295.911522:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346162816:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826295.911909:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346162944:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826295.914912:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826295.915006:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.915573:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.915657:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346162816 00000100:00100000:3.0:1787826295.915668:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346162816 00000100:00100000:3.0:1787826295.915734:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346162944 00000100:00100000:3.0:1787826295.915738:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346162944 00000100:00100000:3.0:1787826295.915781:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11043 00000100:00100000:3.0:1787826295.915796:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346162816:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826295.915815:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826295.915849:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4a0e5 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e47:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a0de expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.915870:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e47:0x0].0x0 (ffff9f7967ac8b00) 00010000:00010000:3.0:1787826295.915884:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826295.915955:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346162816:12345-0@lo:103:ldlm_bl.0 Request processed in 163us (954us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826295.915964:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11043 00000100:00100000:3.0:1787826295.915987:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11044 00000100:00100000:3.0:1787826295.915995:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346162944:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826295.916004:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826295.916018:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a0c9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x18f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a0c2 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.916030:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f79462e8600) 00010000:00010000:3.0:1787826295.916036:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826295.916079:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3655:x1874663346162944:12345-0@lo:103:ldlm_bl.0 Request processed in 86us (507us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826295.916085:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11044 00000100:00100000:3.0:1787826295.916161:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346162944:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826295.916243:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826295.916291:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826295.916314:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332559360 00000100:00100000:3.0:1787826295.916319:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332559360 00000100:00100000:3.0:1787826295.916361:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11045 00000100:00100000:3.0:1787826295.916372:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332559360:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826295.916382:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826295.916397:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f49f25 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801177 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.916409:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:3.0:1787826295.916413:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826295.916461:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332559360:12345-192.168.203.56@tcp:103:rm.0 Request processed in 91us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826295.916479:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11045 00000100:00100000:3.0:1787826295.916501:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332559616 00000100:00100000:3.0:1787826295.916505:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332559616 00000100:00100000:3.0:1787826295.916532:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9323 00000100:00100000:3.0:1787826295.916538:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224272:x1874663332559616:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:3.0:1787826295.916770:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:224272:x1874663332559616:12345-192.168.203.56@tcp:35:rm.0 Request processed in 231us (530us total) trans 4294985365 rc 0/0 00000100:00100000:3.0:1787826295.916777:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9323 00000100:00100000:2.0:1787826295.916866:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346162816:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826295.920886:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.926358:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332559872 00000100:00100000:2.0:1787826295.926526:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332559872 00000100:00100000:2.0:1787826295.926619:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68325 00000100:00100000:2.0:1787826295.926632:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:224272:x1874663332559872:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:2.0:1787826295.926714:0:35098:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826295.926753:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a08a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e46:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011e7 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.926794:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a052 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011cb expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.926809:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f49f33 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080117e expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.926822:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1800) 00010000:00010000:2.0:1787826295.926828:0:35098:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826295.926891:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.926902:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.926914:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.926944:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163072:0@lo:101:mdt00_013.0 00000100:00100000:2.0:1787826295.927041:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826295.927241:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826295.927271:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346163072 00000100:00100000:2.0:1787826295.927285:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346163072 00000100:00100000:2.0:1787826295.927325:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23479 00000100:00100000:2.0:1787826295.927333:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346163072:12345-0@lo:101:mdt00_013.0 00010000:00010000:2.0:1787826295.927349:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826295.927365:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a0fa expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.931783:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.931800:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.931811:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.931863:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.931881:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 7 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.931900:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346163200:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826295.932018:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.932029:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 6 pid: 35093 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.932039:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346163328:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826295.932069:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826295.939235:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826295.939290:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332560000 00000100:00100000:0.0:1787826295.939302:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332560000 00010000:00010000:0.0:1787826295.939370:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 8 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826295.939414:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21894 00000100:00100000:0.0:1787826295.939426:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332560000:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826295.939444:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826295.939452:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 8 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.939469:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 8 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.939497:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826295.939506:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.939525:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826295.939592:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332560000:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 168us (371us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826295.939600:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21894 00000100:00100000:0.0:1787826295.940488:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00080000:1.0:1787826295.940919:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787826295.940976:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f49f72 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080119a expref: 7 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.940998:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346163200:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826295.941012:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346163328:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826295.941057:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.941069:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826295.941072:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826295.941076:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.941085:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:1.0:1787826295.941185:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35098:x1874663346163072:12345-0@lo:101:mdt00_013.0 Request processed in 13852us (14143us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.941205:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23479 00000100:00100000:0.0:1787826295.941587:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163072:0@lo:101:mdt00_013.0 00010000:00010000:0.0:1787826295.941610:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.941623:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.942487:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332560128 00000100:00100000:2.0:1787826295.942507:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332560128 00010000:00010000:2.0:1787826295.942648:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 7 pid: 35093 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.942709:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21895 00000100:00100000:2.0:1787826295.942726:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd80450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332560128:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826295.942743:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826295.942753:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 7 pid: 35093 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.942790:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 7 pid: 35093 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.942818:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f49f09 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801170 expref: 7 pid: 35093 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.942845:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826295.942852:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.942865:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.942876:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.942916:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.942937:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346163456:0@lo:105: 00000100:00100000:2.0:1787826295.943005:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826295.943066:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826295.943068:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346163456 00000100:00100000:1.0:1787826295.943081:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346163456 00000100:00100000:0.0:1787826295.944729:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7834 00000100:00100000:0.0:1787826295.944751:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346163456:12345-0@lo:105: 00010000:00010000:0.0:1787826295.944783:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.944798:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.944809:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.944843:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.944858:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.944867:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.944909:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eee00) 00000100:00100000:0.0:1787826295.945019:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346163456:12345-0@lo:105: Request processed in 275us (2013us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826295.945028:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7834 00000100:00100000:0.0:1787826295.945123:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346163456:0@lo:105: 00010000:00010000:0.0:1787826295.945158:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826295.945214:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd80450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332560128:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2494us (4730us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826295.945223:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21895 00010000:00010000:1.0:1787826295.945731:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.945744:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826295.945784:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826295.945838:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.945847:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.945860:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.945888:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163584:0@lo:101:mdt00_013.0 00000100:00100000:1.0:1787826295.945937:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.945967:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f253c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826295.946027:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346163584 00000100:00100000:0.0:1787826295.946034:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346163584 00000100:00100000:0.0:1787826295.946090:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23480 00000100:00100000:0.0:1787826295.946100:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35098:x1874663346163584:12345-0@lo:101:mdt00_013.0 00010000:00010000:0.0:1787826295.946241:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826295.946262:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a10f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a108 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.946290:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a10f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a108 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.946313:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a10f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a108 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.946328:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826295.946331:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826295.946335:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826295.946814:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35098:x1874663346163584:12345-0@lo:101:mdt00_013.0 Request processed in 711us (871us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826295.946822:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23480 00000100:00100000:1.0:1787826295.948639:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163584:0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826295.948678:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.948695:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826295.948724:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826295.949872:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835180 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163712:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826295.949957:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826295.950000:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826295.950031:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346163712 00000100:00100000:0.0:1787826295.950038:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346163712 00000100:00100000:0.0:1787826295.950093:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23481 00000100:00100000:0.0:1787826295.950126:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35098:x1874663346163712:12345-0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826295.951714:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35098:x1874663346163712:12345-0@lo:1000:mdt00_013.0 Request processed in 1598us (1743us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826295.951730:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23481 00000100:00100000:1.0:1787826295.951902:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835180 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163712:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826295.951999:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163840:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826295.952047:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.953649:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826295.955199:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346163840 00000100:00100000:2.0:1787826295.955215:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346163840 00000100:00100000:2.0:1787826295.955293:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23482 00000100:00100000:2.0:1787826295.955309:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35098:x1874663346163840:12345-0@lo:1000:mdt00_013.0 00000100:00100000:2.0:1787826295.955838:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35098:x1874663346163840:12345-0@lo:1000:mdt00_013.0 Request processed in 534us (3790us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826295.955846:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23482 00000100:00100000:2.0:1787826295.956484:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163840:0@lo:1000:mdt00_013.0 00010000:00010000:2.0:1787826295.959538:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.960445:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.960462:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.960494:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163968:0@lo:101:mdt00_013.0 00000100:00100000:2.0:1787826295.960639:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826295.960690:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826295.960719:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346163968 00000100:00100000:1.0:1787826295.960736:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346163968 00000100:00100000:1.0:1787826295.960847:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23483 00000100:00100000:1.0:1787826295.960865:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:35098:x1874663346163968:12345-0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826295.960892:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826295.960918:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a116 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.960958:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.960977:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a116 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.961020:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.961037:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 6 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.961052:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346164096:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826295.962019:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826295.964047:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346164096:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826295.964130:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a116 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.964155:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826295.964160:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826295.964166:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a116 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.964179:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826295.964256:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:35098:x1874663346163968:12345-0@lo:101:mdt00_013.0 Request processed in 3398us (3617us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.964268:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23483 00000100:00100000:2.0:1787826295.964328:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346163968:0@lo:101:mdt00_013.0 00010000:00010000:2.0:1787826295.964359:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.964378:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.965887:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826295.976692:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332560256 00000100:00100000:3.0:1787826295.978729:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332560256 00010000:00010000:3.0:1787826295.978842:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 7 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.978981:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21896 00000100:00100000:3.0:1787826295.978997:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332560256:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826295.979018:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826295.979028:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 7 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.979057:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 7 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.979087:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f49f8e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011a1 expref: 7 pid: 35080 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.979131:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:3.0:1787826295.979141:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a116 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.979154:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a116 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.979164:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a116 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.979269:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a116 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.979296:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346164224:0@lo:105: 00000100:00100000:3.0:1787826295.979395:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826295.979482:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826295.979891:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346164224 00000100:00100000:3.0:1787826295.979900:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346164224 00000100:00100000:0.0:1787826295.981173:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7835 00000100:00100000:0.0:1787826295.981195:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346164224:12345-0@lo:105: 00010000:00010000:0.0:1787826295.981214:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.981227:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.981237:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.981247:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.981256:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.981267:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826295.981297:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ee000) 00000100:00100000:0.0:1787826295.981393:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346164224:12345-0@lo:105: Request processed in 206us (1999us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826295.981401:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7835 00000100:00100000:3.0:1787826295.982688:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346164224:0@lo:105: 00010000:00010000:3.0:1787826295.982724:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826295.982820:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332560256:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3825us (16942us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826295.982833:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21896 00010000:00010000:3.0:1787826295.982930:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826295.985936:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.985957:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.986016:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.986358:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.986371:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.986394:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,1 mode: --/EX res: [0x240000400:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.986406:0:35098:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.986415:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 4/0,1 mode: --/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.986457:0:35098:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.986479:0:35098:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 11 pid: 24703 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.986497:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001_UUID:35098:1874663346164352:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826295.987722:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826295.993620:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826295.993698:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001_UUID:35098:1874663346164352:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826295.993701:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332560384 00000100:00100000:3.0:1787826295.993714:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332560384 00010000:00010000:2.0:1787826295.993739:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,1 mode: --/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.993781:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 12 pid: 24703 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.993866:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21897 00000100:00100000:3.0:1787826295.993879:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332560384:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826295.993898:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826295.993907:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 12 pid: 24703 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.993923:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 12 pid: 24703 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.993949:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a060 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011d2 expref: 12 pid: 24703 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.993969:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7967ac1100) 00010000:00010000:3.0:1787826295.993976:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,1 mode: --/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.993986:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.993994:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826295.994029:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826295.994144:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332560384:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 264us (534us total) trans 0 rc 0/0 00010000:00010000:2.0:1787826295.994145:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826295.994165:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21897 00010000:00010000:2.0:1787826295.994167:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.994229:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.994237:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826295.994249:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826295.994272:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346164480:0@lo:101:mdt00_013.0 00000100:00100000:2.0:1787826295.994361:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.994405:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346164480 00000100:00100000:1.0:1787826295.994413:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346164480 00000100:00100000:2.0:1787826295.994416:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826295.994469:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23484 00000100:00100000:1.0:1787826295.994479:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:35098:x1874663346164480:12345-0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826295.994499:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826295.994521:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a139 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a132 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.994545:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a139 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a132 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.994560:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a139 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a132 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.994569:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:1.0:1787826295.994571:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826295.994574:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826295.994625:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:35098:x1874663346164480:12345-0@lo:101:mdt00_013.0 Request processed in 148us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826295.994631:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23484 00000100:00100000:1.0:1787826295.994677:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346164480:0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826295.994695:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826295.994707:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826295.997016:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346164608:0@lo:403:mdt00_013.0 00000100:00100000:1.0:1787826295.997152:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826295.997200:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826295.997296:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346164608 00000100:00100000:2.0:1787826295.997311:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346164608 00000100:00100000:2.0:1787826295.997393:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23485 00000100:00100000:2.0:1787826295.997405:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:35098:x1874663346164608:12345-0@lo:403:mdt00_013.0 00000100:00100000:3.0:1787826296.007978:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:35098:x1874663346164608:12345-0@lo:403:mdt00_013.0 Request processed in 10560us (10811us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826296.007998:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23485 00000100:00100000:1.0:1787826296.008189:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346164608:0@lo:403:mdt00_013.0 00000040:00080000:1.0:1787826296.009528:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1188, 952 off1080736 00000040:00080000:1.0:1787826296.009792:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1507, 1032 off1521688 00000004:00080000:1.0:1787826296.009801:0:35098:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852f25180 with version 6806 00000004:00080000:3.0:1787826296.009986:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6806 rpc_version 6806 00000020:00100000:3.0:1787826296.010040:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826296.010053:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784c6b8e00 x1874663346164864/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2608/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826296.010084:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346164864:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826296.010198:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.010247:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826296.011327:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346164864 00000100:00100000:1.0:1787826296.011341:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346164864 00000100:00100000:1.0:1787826296.011418:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23486 00000100:00100000:1.0:1787826296.011431:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346164864:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826296.016041:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346164864:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826296.016302:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346164864:12345-0@lo:1000:osp_up0-1.0 Request processed in 4377us (5607us total) trans 8589957352 rc 0/0 00000100:00100000:1.0:1787826296.016316:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23486 00000004:00200000:2.0:1787826296.018944:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e46:0x0]: returning size 4096 00010000:00010000:2.0:1787826296.018987:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019001:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019009:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019017:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826296.019022:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019030:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019374:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x191:0x0].0x0 (ffff9f7967ac1100) 00010000:00010000:2.0:1787826296.019379:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826296.019386:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 2/0,0 mode: --/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019393:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a12b lrc: 0/0,0 mode: --/EX res: [0x240000400:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019426:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019436:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019444:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019603:0:35098:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826296.019616:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x18f:0x0] 00010000:00010000:2.0:1787826296.019627:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019634:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019640:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019646:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826296.019649:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019655:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019666:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1800) 00010000:00010000:2.0:1787826296.019669:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826296.019672:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019678:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a124 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019688:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019696:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.019702:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.023658:0:35098:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826296.023694:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826296.023706:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.023721:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.023730:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.023787:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787826296.023805:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826296.023814:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.023823:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826296.023829:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826296.023839:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.023898:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.023906:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826296.023948:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.023957:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee000/0xe21309aa81f4a116 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a11d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.023986:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826296.023990:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.023997:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826296.024001:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826296.024008:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.024025:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.024032:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826296.024062:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.024075:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826296.024077:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826296.024079:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826296.024102:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346165120:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826296.025005:0:35098:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:2.0:1787826296.025016:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a0fa lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a101 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826296.025045:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826296.025054:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.025063:0:35098:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826296.025262:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224272:x1874663332559872:12345-192.168.203.56@tcp:36:rm.0 Request processed in 98626us (104386us total) trans 4294985366 rc 0/0 00000100:00100000:2.0:1787826296.025272:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68325 00010000:00010000:3.0:1787826296.025660:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.025671:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826296.025678:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826296.025689:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.025730:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.025743:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826296.025766:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.025774:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee600/0xe21309aa81f4a132 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a139 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.025796:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826296.025798:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826296.025800:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826296.026413:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826296.026490:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826296.026511:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346165248:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826296.026572:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826296.026896:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346164992:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826296.026937:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826296.027493:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346165120 00000100:00100000:2.0:1787826296.027505:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346165120 00000100:00100000:2.0:1787826296.027569:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346165248 00000100:00100000:2.0:1787826296.027573:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346165248 00000100:00100000:3.0:1787826296.027717:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826296.027790:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346164992 00000100:00100000:1.0:1787826296.027795:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346164992 00000100:00100000:1.0:1787826296.027841:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11046 00000100:00100000:1.0:1787826296.027853:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+17:3654:x1874663346165120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826296.027870:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826296.027898:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a101 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a0fa expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.027919:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826296.027925:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826296.027926:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11047 00000100:00100000:2.0:1787826296.027939:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+16:3654:x1874663346165248:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826296.027949:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826296.027982:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+16:3654:x1874663346165120:12345-0@lo:103:ldlm_bl.0 Request processed in 131us (1573us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826296.027991:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11046 00000100:00100000:1.0:1787826296.028010:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11048 00000100:00100000:1.0:1787826296.028017:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+15:3653:x1874663346164992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826296.028024:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826296.028040:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a11d lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e46:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a116 expref: 15 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.028040:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a139 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x18f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a132 expref: 15 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.028059:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:2.0:1787826296.028069:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826296.028100:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+14:3654:x1874663346165248:12345-0@lo:103:ldlm_bl.0 Request processed in 166us (1529us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.028125:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11047 00010000:00010000:1.0:1787826296.028195:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826296.028202:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826296.028341:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346164992:12345-0@lo:103:ldlm_bl.0 Request processed in 324us (625us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826296.028348:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11048 00000100:00100000:1.0:1787826296.028412:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346165248:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826296.028425:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346165120:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826296.031594:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346164992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826296.132730:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.132910:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332560512 00000100:00100000:2.0:1787826296.132923:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332560512 00000100:00100000:2.0:1787826296.133002:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68326 00000100:00100000:2.0:1787826296.133015:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224280:x1874663332560512:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826296.133076:0:6528:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:2.0:1787826296.133217:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224280:x1874663332560512:12345-192.168.203.56@tcp:41:stat.0 Request processed in 204us (501us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.133230:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68326 00000100:00100000:0.0:1787826296.185242:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826296.186865:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332560640 00000100:00100000:1.0:1787826296.186886:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332560640 00000100:00100000:1.0:1787826296.186992:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68327 00000100:00100000:1.0:1787826296.187006:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224284:x1874663332560640:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826296.187033:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826296.187061:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a140 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608011fc expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.187189:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.187207:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.187221:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826296.187282:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826296.187288:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826296.187312:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.187342:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a140 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.187365:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.187376:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826296.187379:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826296.187384:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826296.187474:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224284:x1874663332560640:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 470us (2256us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826296.187481:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68327 00000100:00100000:1.0:1787826296.195146:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826296.195273:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332560768 00000100:00100000:3.0:1787826296.195287:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332560768 00000100:00100000:3.0:1787826296.195368:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68328 00000100:00100000:3.0:1787826296.195383:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224284:x1874663332560768:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826296.195409:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826296.195436:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a14e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801203 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.195519:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.195536:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.195550:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.195567:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.195579:0:35092:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a10f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a108 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.195589:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.195634:0:35092:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a10f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a108 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826296.195664:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346165376:0@lo:104: 00000100:00100000:3.0:1787826296.195726:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.195789:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.195907:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346165376 00000100:00100000:0.0:1787826296.195914:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346165376 00000100:00100000:0.0:1787826296.195963:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7836 00000100:00100000:0.0:1787826296.195972:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:35092:x1874663346165376:12345-0@lo:104: 00010000:00010000:0.0:1787826296.195986:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.196054:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826296.196061:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:35092:x1874663346165376:12345-0@lo:104: Request processed in 91us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826296.196068:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7836 00010000:00010000:0.0:1787826296.196093:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826296.196100:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.196137:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826296.196144:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826296.196154:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.196188:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.196197:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826296.196218:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.196226:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a108 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a10f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.196250:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826296.196252:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826296.196254:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826296.196337:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346165504:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826296.196379:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.196501:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346165376:0@lo:104: 00010000:00010000:3.0:1787826296.196524:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826296.196581:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346165504 00000100:00100000:0.0:1787826296.196587:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346165504 00010000:00010000:0.0:1787826296.196608:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a10f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a108 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826296.196648:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21898 00000100:00100000:0.0:1787826296.196657:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346165504:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826296.196668:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1787826296.205982:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957352 is committed 00010000:00010000:2.0:1787826296.211708:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a10f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a108 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.211731:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5b00) 00010000:00010000:2.0:1787826296.211736:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.211745:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.211763:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.211850:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826296.211975:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346165504:12345-0@lo:103:ldlm_bl.0 Request processed in 15318us (15594us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.211985:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21898 00010000:00010000:3.0:1787826296.212035:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212043:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212205:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212213:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212220:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212226:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212232:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826296.212237:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212244:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212255:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826296.212257:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826296.212261:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212267:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a155 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212279:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212284:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212290:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212295:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826296.212298:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212304:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212311:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5b00) 00010000:00010000:3.0:1787826296.212313:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826296.212315:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212321:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a15c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212359:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a14e lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801203 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212367:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826296.212369:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826296.212373:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a14e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801203 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.212382:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68800, rc 0) 00000100:00100000:3.0:1787826296.212609:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224284:x1874663332560768:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 17227us (17500us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826296.212617:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68328 00000100:00100000:1.0:1787826296.212762:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346165504:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826296.227397:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826296.227605:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332560896 00000100:00100000:3.0:1787826296.227616:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332560896 00000100:00100000:3.0:1787826296.227676:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68329 00000100:00100000:3.0:1787826296.227686:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224284:x1874663332560896:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826296.227706:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826296.227727:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a163 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080120a expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.227805:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.227818:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.227829:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826296.227877:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826296.227881:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826296.227902:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.227926:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a163 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.227946:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.227965:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826296.227967:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826296.227971:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826296.228036:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224284:x1874663332560896:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 351us (649us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826296.228052:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68329 00000100:00100000:1.0:1787826296.241020:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.241292:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332561152 00000100:00100000:2.0:1787826296.241303:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332561152 00000100:00100000:2.0:1787826296.241436:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68330 00000100:00100000:2.0:1787826296.241448:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224284:x1874663332561152:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:2.0:1787826296.241564:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346165632:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826296.241619:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826296.241660:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.241860:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346165632 00000100:00100000:0.0:1787826296.241866:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346165632 00000100:00100000:0.0:1787826296.241905:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23487 00000100:00100000:0.0:1787826296.241913:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346165632:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826296.241986:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346165632:12345-0@lo:1000:mdt00_000.0 Request processed in 74us (366us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826296.241994:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23487 00000100:00100000:2.0:1787826296.242242:0:6527:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957352 gen 2 00000004:00080000:2.0:1787826296.242248:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957350 committed transno 8589957352 00000004:00080000:2.0:1787826296.242269:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957351 committed transno 8589957352 00000004:00080000:2.0:1787826296.242282:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957352 committed transno 8589957352 00000100:00100000:2.0:1787826296.242297:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346165632:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826296.242341:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346165760:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826296.242372:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826296.242395:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826296.242524:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346165760 00000100:00100000:3.0:1787826296.242530:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346165760 00000100:00100000:3.0:1787826296.242570:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23488 00000100:00100000:3.0:1787826296.242579:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346165760:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826296.242672:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346165760:12345-0@lo:1000:mdt00_000.0 Request processed in 95us (298us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787826296.242680:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23488 00000100:00100000:2.0:1787826296.242790:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346165760:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787826296.242838:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.242848:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.242858:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826296.242870:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346165888:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787826296.242894:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826296.242918:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.242970:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346165888 00000100:00100000:0.0:1787826296.242975:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346165888 00000100:00100000:0.0:1787826296.243008:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23489 00000100:00100000:0.0:1787826296.243015:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346165888:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787826296.243024:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826296.243039:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a171 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.243074:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.243091:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.243099:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.243159:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.243172:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 6 pid: 35094 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826296.243185:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346166016:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826296.243257:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.243267:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 6 pid: 6529 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826296.243276:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346166144:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826296.243297:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.245685:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346166144:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826296.245985:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.246087:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332561280 00000100:00100000:2.0:1787826296.246096:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332561280 00010000:00010000:2.0:1787826296.246172:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 7 pid: 6529 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826296.246202:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21899 00000100:00100000:2.0:1787826296.246212:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd82050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332561280:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826296.246225:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826296.246232:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 7 pid: 6529 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.246244:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 7 pid: 6529 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.246265:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a147 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608011fc expref: 7 pid: 6529 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.246282:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826296.246288:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.246298:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826296.246355:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd82050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332561280:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 144us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.246362:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21899 00000100:00100000:0.0:1787826296.247248:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346166016:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826296.247281:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.247292:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826296.247295:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826296.247299:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.247307:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826296.247360:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346165888:12345-0@lo:101:mdt00_000.0 Request processed in 4344us (4464us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826296.247369:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23489 00000100:00100000:2.0:1787826296.247494:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346165888:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787826296.247510:0:6527:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.247521:0:6527:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826296.248463:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826296.248466:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826296.248601:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332561408 00000100:00100000:0.0:1787826296.248611:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332561408 00010000:00010000:0.0:1787826296.248650:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 7 pid: 35094 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826296.248687:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21900 00000100:00100000:0.0:1787826296.248697:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332561408:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826296.248712:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826296.248720:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 7 pid: 35094 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.248732:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 7 pid: 35094 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.248762:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a16a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080120a expref: 7 pid: 35094 timeout: 8208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.248779:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826296.248786:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.248797:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.248807:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.248839:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826296.248857:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346166272:0@lo:105: 00000100:00100000:0.0:1787826296.248910:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826296.248958:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826296.249022:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346166272 00000100:00100000:3.0:1787826296.249027:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346166272 00000100:00100000:3.0:1787826296.249070:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7837 00000100:00100000:3.0:1787826296.249086:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346166272:12345-0@lo:105: 00010000:00010000:3.0:1787826296.249097:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.249135:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.249144:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.249153:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.249161:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.249176:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.249203:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eee00) 00000100:00100000:3.0:1787826296.249252:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346166272:12345-0@lo:105: Request processed in 173us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826296.249259:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7837 00000100:00100000:0.0:1787826296.249475:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346166272:0@lo:105: 00010000:00010000:0.0:1787826296.249493:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826296.249547:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332561408:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 852us (1092us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826296.249554:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21900 00010000:00010000:2.0:1787826296.249688:0:6527:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.249699:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826296.249728:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826296.249774:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.249782:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.249792:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826296.249809:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166400:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787826296.249846:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826296.249877:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.250228:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346166400 00000100:00100000:0.0:1787826296.250234:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346166400 00000100:00100000:0.0:1787826296.250278:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23490 00000100:00100000:0.0:1787826296.250289:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346166400:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787826296.250304:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826296.250320:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a186 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a17f expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.250341:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a186 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a17f expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.250352:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a186 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a17f expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.250359:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5700) 00010000:00010000:0.0:1787826296.250362:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826296.250364:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826296.250406:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166400:12345-0@lo:101:mdt00_000.0 Request processed in 119us (558us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826296.250413:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23490 00000100:00100000:3.0:1787826296.250542:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166400:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787826296.250579:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.250595:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826296.250622:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826296.250699:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166528:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826296.250778:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.250817:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.250872:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346166528 00000100:00100000:0.0:1787826296.250879:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346166528 00000100:00100000:0.0:1787826296.250921:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23491 00000100:00100000:0.0:1787826296.250928:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166528:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826296.251017:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166528:12345-0@lo:1000:mdt00_000.0 Request processed in 89us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826296.251024:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23491 00000100:00100000:3.0:1787826296.251310:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166528:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826296.251378:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166656:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826296.251412:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.251512:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.252654:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346166656 00000100:00100000:0.0:1787826296.252664:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346166656 00000100:00100000:0.0:1787826296.252772:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23492 00000100:00100000:0.0:1787826296.252782:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166656:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826296.252914:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166656:12345-0@lo:1000:mdt00_000.0 Request processed in 133us (1500us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826296.252921:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23492 00000100:00100000:1.0:1787826296.253000:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166656:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826296.253141:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166784:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826296.253176:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.254834:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346166784 00000100:00100000:3.0:1787826296.254846:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346166784 00000100:00100000:3.0:1787826296.254901:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23493 00000100:00100000:3.0:1787826296.254910:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166784:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826296.255013:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166784:12345-0@lo:1000:mdt00_000.0 Request processed in 104us (1836us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787826296.255020:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23493 00000100:00100000:1.0:1787826296.255332:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826296.255353:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166784:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826296.255426:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166912:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826296.255524:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826296.255544:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826296.255595:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346166912 00000100:00100000:3.0:1787826296.255599:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346166912 00000100:00100000:3.0:1787826296.255708:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23494 00000100:00100000:3.0:1787826296.255716:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166912:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826296.255812:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346166912:12345-0@lo:1000:mdt00_000.0 Request processed in 97us (287us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826296.255818:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23494 00000100:00100000:0.0:1787826296.255930:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346166912:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826296.255973:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346167040:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826296.255997:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826296.256052:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826296.256237:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346167040 00000100:00100000:3.0:1787826296.256242:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346167040 00000100:00100000:3.0:1787826296.256309:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23495 00000100:00100000:3.0:1787826296.256316:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346167040:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826296.256373:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346167040:12345-0@lo:1000:mdt00_000.0 Request processed in 58us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826296.256380:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23495 00000100:00100000:0.0:1787826296.256571:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346167040:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826296.256617:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346167168:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826296.256643:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826296.256663:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826296.258528:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346167168 00000100:00100000:3.0:1787826296.258542:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346167168 00000100:00100000:3.0:1787826296.258605:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23496 00000100:00100000:3.0:1787826296.258615:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346167168:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826296.258724:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346167168:12345-0@lo:1000:mdt00_000.0 Request processed in 110us (2079us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826296.258732:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23496 00000100:00100000:0.0:1787826296.259125:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346167168:0@lo:1000:mdt00_000.0 00000040:00080000:0.0:1787826296.260478:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1190, 2168 off1083512 00000040:00080000:0.0:1787826296.260566:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1508, 2368 off1524056 00000004:00080000:0.0:1787826296.260572:0:6527:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7851057180 with version 6807 00000004:00080000:3.0:1787826296.261008:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6807 rpc_version 6807 00000020:00100000:3.0:1787826296.261061:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826296.261072:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5c8000 x1874663346167424/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826296.261233:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346167424:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826296.261307:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.261337:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826296.261419:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346167424 00000100:00100000:2.0:1787826296.261435:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346167424 00000100:00100000:2.0:1787826296.261481:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23497 00000100:00100000:2.0:1787826296.261490:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346167424:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826296.267007:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346167424:12345-0@lo:1000:osp_up0-1.0 Request processed in 5508us (5690us total) trans 8589957353 rc 0/0 00000100:00100000:2.0:1787826296.267021:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23497 00000100:00100000:3.0:1787826296.269440:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346167424:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826296.269667:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.269684:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.269696:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.269719:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.269726:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.269732:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.269803:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.269812:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.269824:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826296.269858:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346167552:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787826296.269927:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826296.269976:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78510579c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826296.270693:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346167552 00000100:00100000:2.0:1787826296.270704:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346167552 00000100:00100000:2.0:1787826296.270771:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23498 00000100:00100000:2.0:1787826296.270782:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6527:x1874663346167552:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787826296.270804:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826296.270824:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a1a2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a19b expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.270864:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a1a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a19b expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.270877:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a1a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a19b expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.270886:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f7967ac8e00) 00010000:00010000:2.0:1787826296.270889:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826296.270892:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826296.270955:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6527:x1874663346167552:12345-0@lo:101:mdt00_000.0 Request processed in 175us (1030us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.270962:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23498 00000100:00100000:0.0:1787826296.271332:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346167552:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787826296.271350:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271361:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826296.271386:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000087f0691 reply state 000000001c0df285 transno 4294985367 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271400:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271406:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826296.271409:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826296.271412:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826296.271418:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271437:0:6527:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826296.271450:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x190:0x0] 00000004:00010000:0.0:1787826296.271457:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000087f0691 reply state 000000001c0df285 transno 4294985367 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271465:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271471:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:0.0:1787826296.271474:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826296.271476:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787826296.271482:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e48:0x0]: returning size 4096 00000004:00080000:0.0:1787826296.271493:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826296.271500:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271507:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271514:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271541:0:6527:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826296.271546:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826296.271550:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.271558:0:6527:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826296.271641:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224284:x1874663332561152:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 30194us (30629us total) trans 4294985367 rc 0/0 00000100:00100000:0.0:1787826296.271648:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68330 00010000:00010000:2.0:1787826296.272187:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826296.272194:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.272203:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826296.272209:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826296.272217:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.272249:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.272256:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826296.272289:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.272298:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a171 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a178 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.272311:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826296.272313:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826296.272315:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826296.272402:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826296.272419:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346167680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826296.272474:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826296.272608:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346167680 00000100:00100000:2.0:1787826296.272615:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346167680 00000100:00100000:2.0:1787826296.272668:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11049 00000100:00100000:2.0:1787826296.272677:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346167680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826296.272689:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826296.272708:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a178 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a171 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.272721:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826296.272725:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826296.272772:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346167680:12345-0@lo:103:ldlm_bl.0 Request processed in 97us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.272778:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11049 00000100:00100000:2.0:1787826296.272812:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346167680:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826296.280505:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.280614:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332561536 00000100:00100000:2.0:1787826296.280631:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332561536 00000100:00100000:2.0:1787826296.280718:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68331 00000100:00100000:2.0:1787826296.280729:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224273:x1874663332561536:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826296.280749:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826296.280783:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a1a9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801211 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.280862:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4a1b0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.280875:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4a1b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.280886:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4a1b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826296.280931:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826296.280935:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826296.280955:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4a1b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.280977:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a1a9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801211 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.280997:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4a1b0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801211 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.281005:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826296.281009:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826296.281012:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826296.281259:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224273:x1874663332561536:12345-192.168.203.56@tcp:101:bash.0 Request processed in 530us (766us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.281268:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68331 00000100:00100000:1.0:1787826296.285639:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.285737:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332561664 00000100:00100000:2.0:1787826296.285749:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332561664 00000100:00100000:2.0:1787826296.285825:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68332 00000100:00100000:2.0:1787826296.285835:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224273:x1874663332561664:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826296.285866:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826296.285890:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a1b7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801218 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.285954:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.285969:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.285980:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.285990:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.286001:0:17813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a186 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a17f expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.286008:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.286044:0:17813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a186 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a17f expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826296.286061:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346167808:0@lo:104: 00000100:00100000:2.0:1787826296.286151:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826296.286204:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.286230:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346167808 00000100:00100000:0.0:1787826296.286244:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346167808 00000100:00100000:0.0:1787826296.286319:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7838 00000100:00100000:0.0:1787826296.286333:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17813:x1874663346167808:12345-0@lo:104: 00010000:00010000:0.0:1787826296.286353:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826296.286589:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826296.286596:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17813:x1874663346167808:12345-0@lo:104: Request processed in 268us (445us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826296.286606:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7838 00010000:00010000:3.0:1787826296.287428:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826296.287439:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.287451:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826296.287458:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826296.287468:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.287513:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.287520:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826296.287568:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.287576:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4a17f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a186 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.287605:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826296.287606:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826296.287608:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826296.287739:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850359880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346167936:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826296.287820:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.287932:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346167936 00000100:00100000:3.0:1787826296.287941:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346167936 00010000:00010000:3.0:1787826296.287972:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a186 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a17f expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826296.288014:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21901 00000100:00100000:3.0:1787826296.288034:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346167936:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826296.288048:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787826296.288385:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346167808:0@lo:104: 00010000:00010000:2.0:1787826296.288416:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826296.303216:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957353 is committed 00010000:00010000:2.0:1787826296.305385:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a186 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a17f expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.305407:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5700) 00010000:00010000:2.0:1787826296.305413:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.305422:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.305429:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.305468:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826296.305609:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346167936:12345-0@lo:103:ldlm_bl.0 Request processed in 17576us (17787us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.305619:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21901 00010000:00010000:3.0:1787826296.305791:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.305808:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826296.305993:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035b800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:17813:1874663346168064:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787826296.306066:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.306111:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753b3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.306162:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346168064 00000100:00100000:0.0:1787826296.306172:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346168064 00000100:00100000:0.0:1787826296.306230:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23499 00000100:00100000:0.0:1787826296.306614:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:17813:x1874663346168064:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826296.306722:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:17813:x1874663346168064:12345-0@lo:1000:mdt00_005.0 Request processed in 113us (655us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826296.306740:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23499 00000100:00100000:1.0:1787826296.306868:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850359880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346167936:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826296.307171:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035b800 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:17813:1874663346168064:0@lo:1000:mdt00_005.0 00010000:00010000:3.0:1787826296.307530:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a1cc lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307550:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a1cc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307562:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a1cc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826296.307592:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a1cc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307611:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307618:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307625:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307633:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307640:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826296.307644:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307652:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307665:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826296.307668:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826296.307673:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307679:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a1be lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307696:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307708:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307715:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307720:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826296.307723:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307730:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307737:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5700) 00010000:00010000:3.0:1787826296.307739:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826296.307742:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307764:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a1c5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307826:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a1b7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801218 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307842:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a1cc lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801218 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.307851:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:3.0:1787826296.307854:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826296.307857:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68800, rc 0) 00000100:00100000:3.0:1787826296.307950:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224273:x1874663332561664:12345-192.168.203.56@tcp:101:bash.0 Request processed in 22115us (22321us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826296.307959:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68332 00000100:00100000:0.0:1787826296.311188:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.311362:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332561792 00000100:00100000:2.0:1787826296.311375:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332561792 00000100:00100000:2.0:1787826296.311434:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68333 00000100:00100000:2.0:1787826296.311444:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224273:x1874663332561792:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826296.311469:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826296.311490:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a1d3 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080121f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.311574:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a1da lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.311589:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a1da lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.311601:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a1da lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826296.311703:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e48:0x0]: returning size 4096 00000004:00200000:2.0:1787826296.311708:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826296.311746:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a1d3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080121f expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.311779:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a1da lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080121f expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.311789:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:2.0:1787826296.311792:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826296.311796:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826296.311850:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224273:x1874663332561792:12345-192.168.203.56@tcp:101:bash.0 Request processed in 407us (672us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.311858:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68333 00000100:00100000:1.0:1787826296.319633:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.320834:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332561920 00000100:00100000:2.0:1787826296.320849:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332561920 00000100:00100000:2.0:1787826296.320929:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68334 00000100:00100000:2.0:1787826296.320940:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224273:x1874663332561920:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826296.320961:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826296.320982:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a1e1 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801226 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.321397:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a1e8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.321413:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a1e8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.321424:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a1e8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826296.321473:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x192:0x0]: returning size 4096 00000004:00200000:2.0:1787826296.321478:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826296.321513:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a1e1 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801226 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.321532:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a1e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801226 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.321867:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826296.321870:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826296.321874:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826296.321944:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224273:x1874663332561920:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1005us (2326us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.321952:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68334 00000100:00100000:1.0:1787826296.330437:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.330554:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332562048 00000100:00100000:2.0:1787826296.330570:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332562048 00000100:00100000:2.0:1787826296.330662:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68335 00000100:00100000:2.0:1787826296.330672:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224273:x1874663332562048:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826296.330694:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826296.330716:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a1ef lrc: 2/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080122d expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.330792:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.330807:0:35080:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a1a2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a19b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.330816:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.330850:0:35080:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a1a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a19b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826296.330866:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000_UUID:35080:1874663346168192:0@lo:104: 00000100:00100000:2.0:1787826296.330932:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826296.331023:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826296.336645:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826296.336674:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346168192 00000100:00100000:0.0:1787826296.336684:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346168192 00000100:00100000:1.0:1787826296.336860:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7839 00000100:00100000:1.0:1787826296.336888:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c044050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:35080:x1874663346168192:12345-0@lo:104: 00010000:00010000:1.0:1787826296.336911:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.337241:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826296.337249:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c044050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:35080:x1874663346168192:12345-0@lo:104: Request processed in 375us (6315us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826296.337258:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7839 00010000:00010000:3.0:1787826296.337336:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826296.337350:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.337363:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826296.337372:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826296.337382:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.337427:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.337434:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826296.337499:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346168320:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826296.337630:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826296.337717:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 1/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.337726:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a19b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a1a2 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.337762:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826296.337765:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826296.337767:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826296.337809:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346168320 00000100:00100000:1.0:1787826296.337816:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346168320 00010000:00010000:1.0:1787826296.337843:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a1a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a19b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826296.337879:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21902 00000100:00100000:1.0:1787826296.337890:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346168320:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826296.337903:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826296.338649:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f7967ac8e00) 00010000:00010000:1.0:1787826296.338657:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.338667:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.338674:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826296.338694:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826296.338810:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346168320:12345-0@lo:103:ldlm_bl.0 Request processed in 921us (1179us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826296.338818:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21902 00000100:00100000:0.0:1787826296.338915:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346168320:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826296.342596:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a1a2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a19b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826296.342630:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000_UUID:35080:1874663346168192:0@lo:104: 00010000:00010000:2.0:1787826296.342666:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.342678:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.342685:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826296.342787:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x190:0x0]: returning size 4096 00000004:00200000:2.0:1787826296.342794:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826296.342864:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a1ef lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080122d expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.342881:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080122d expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.342890:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f7967ac8e00) 00010000:00010000:2.0:1787826296.342893:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826296.342897:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826296.342988:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224273:x1874663332562048:12345-192.168.203.56@tcp:101:bash.0 Request processed in 12315us (12886us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.342998:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68335 00000100:00100000:0.0:1787826296.345620:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826296.346298:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332562304 00000100:00100000:3.0:1787826296.346310:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332562304 00000100:00100000:3.0:1787826296.346378:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68336 00000100:00100000:3.0:1787826296.346389:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224273:x1874663332562304:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826296.346410:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826296.346414:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826296.346442:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a1f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080122d expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.346462:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f7967ac8e00) 00010000:00010000:3.0:1787826296.346472:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826296.346488:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a1fd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801234 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.346616:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.346629:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.346639:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.346657:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.346665:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.346672:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826296.347017:0:35094:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x51e:0x0], index 0: 0 00010000:00010000:3.0:1787826296.347527:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a212 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347538:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a212 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347547:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a212 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347579:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347592:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347599:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347606:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347612:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826296.347617:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347625:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.347636:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f7961d81900) 00010000:00010000:3.0:1787826296.347639:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826296.348251:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.348262:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a204 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826296.348278:0:35094:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ee6c4646 reply state 000000002c3a824c transno 8589957354 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.348290:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.348296:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f7961d81200) 00010000:00010000:3.0:1787826296.348298:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826296.348301:0:35094:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826296.348344:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a1fd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801234 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.348367:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a212 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801234 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.348374:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db8:0x0].0x0 (ffff9f7961d81b00) 00010000:00010000:3.0:1787826296.348376:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826296.348379:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826296.348460:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224273:x1874663332562304:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2071us (2848us total) trans 8589957354 rc 0/0 00000100:00100000:3.0:1787826296.348469:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68336 00000100:00100000:1.0:1787826296.361385:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.361970:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332562432 00000100:00100000:2.0:1787826296.361982:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332562432 00000100:00080000:1.0:1787826296.362243:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826296.362420:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30803 00000100:00100000:2.0:1787826296.362433:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224273:x1874663332562432:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826296.362452:0:20167:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826296.362474:0:20167:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080123b expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826296.362500:0:20167:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x51e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826296.362524:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=8108s v=(0 0) 00010000:00010000:2.0:1787826296.362528:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826296.362538:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 7 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.362558:0:20167:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 7 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.362571:0:20167:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826296.362629:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224273:x1874663332562432:12345-192.168.203.56@tcp:101:bash.0 Request processed in 199us (1254us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.362635:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30803 00000100:00100000:1.0:1787826296.376968:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.377073:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332562688 00000100:00100000:2.0:1787826296.377084:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332562688 00000100:00100000:2.0:1787826296.377296:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9324 00000100:00100000:2.0:1787826296.377307:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224273:x1874663332562688:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826296.377604:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224273:x1874663332562688:12345-192.168.203.56@tcp:35:bash.0 Request processed in 299us (644us total) trans 8589957355 rc 0/0 00000100:00100000:2.0:1787826296.377612:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9324 00000100:00100000:0.0:1787826296.392016:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826296.392467:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332562944 00000100:00100000:3.0:1787826296.392477:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332562944 00000100:00100000:3.0:1787826296.393148:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68337 00000100:00100000:3.0:1787826296.393172:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224273:x1874663332562944:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826296.393195:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826296.393198:0:8752:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826296.393226:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a1e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801226 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.393246:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:3.0:1787826296.393251:0:8752:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826296.393266:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4a227 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801242 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.393392:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 3/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.393404:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.393414:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.393429:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.393436:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.393443:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826296.393646:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035a300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346168448:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826296.393706:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826296.393745:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826296.393805:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346168448 00000100:00100000:2.0:1787826296.393812:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346168448 00000100:00100000:2.0:1787826296.393904:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23500 00000100:00100000:2.0:1787826296.393913:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346168448:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787826296.393992:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346168448:12345-0@lo:1000:mdt00_003.0 Request processed in 81us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.394000:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23500 00000100:00100000:3.0:1787826296.394027:0:8752:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957353 gen 2 00000004:00080000:3.0:1787826296.394033:0:8752:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957353 committed transno 8589957353 00000100:00100000:3.0:1787826296.394065:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035a300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346168448:0@lo:1000:mdt00_003.0 00000004:00080000:3.0:1787826296.395829:0:8752:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4c9:0x0], index 1: 0 00010000:00010000:3.0:1787826296.396011:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a23c lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396026:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a23c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396038:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a23c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396074:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396081:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396088:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396095:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396101:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826296.396429:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396439:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396459:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:3.0:1787826296.396461:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826296.396466:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 3/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396472:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a22e lrc: 0/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826296.396488:0:8752:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000047c2c6c1 reply state 000000009c5b42b9 transno 4294985368 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396500:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396506:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x4ec94fd6 (ffff9f7961d81900) 00010000:00010000:3.0:1787826296.396508:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826296.396512:0:8752:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826296.396621:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4a227 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801242 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396637:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a23c lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801242 expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826296.396645:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f7961d81800) 00010000:00010000:3.0:1787826296.396647:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826296.396650:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826296.396781:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224273:x1874663332562944:12345-192.168.203.56@tcp:101:bash.0 Request processed in 3613us (4773us total) trans 4294985368 rc 0/0 00000100:00100000:3.0:1787826296.396790:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68337 00000100:00100000:1.0:1787826296.403958:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.404697:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332563072 00000100:00100000:2.0:1787826296.404709:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332563072 00000100:00100000:2.0:1787826296.404795:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30804 00000100:00100000:2.0:1787826296.404806:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:224273:x1874663332563072:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826296.404823:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826296.404846:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801249 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826296.404872:0:24218:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4c9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826296.404893:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c9:0x0].0x0 since=8108s v=(0 0) 00010000:00010000:2.0:1787826296.404898:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826296.404908:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.404925:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.404937:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826296.404990:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224273:x1874663332563072:12345-192.168.203.56@tcp:101:bash.0 Request processed in 186us (1042us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.404997:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30804 00000100:00080000:1.0:1787826296.405065:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826296.414703:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.414851:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332563328 00000100:00100000:2.0:1787826296.414864:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332563328 00000100:00100000:2.0:1787826296.414930:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9325 00000100:00100000:2.0:1787826296.414942:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224273:x1874663332563328:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826296.415537:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224273:x1874663332563328:12345-192.168.203.56@tcp:35:bash.0 Request processed in 593us (845us total) trans 4294985369 rc 0/0 00000100:00100000:2.0:1787826296.415548:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9325 00000100:00100000:0.0:1787826296.435347:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826296.439806:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826296.443805:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332563456 00000100:00100000:2.0:1787826296.443820:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332563456 00000100:00100000:2.0:1787826296.444410:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332563584 00000100:00100000:2.0:1787826296.444415:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332563584 00000100:00100000:2.0:1787826296.444451:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11050 00000100:00100000:2.0:1787826296.444461:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:49150:x1874663332563456:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826296.444478:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826296.444504:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4a1b0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801211 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.444521:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826296.444530:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a212 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801234 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.444540:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db8:0x0].0x0 (ffff9f7961d81b00) 00010000:00010000:2.0:1787826296.444554:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a1cc lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801218 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.444561:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f7961d81d00) 00010000:00010000:2.0:1787826296.444566:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826296.444621:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:49150:x1874663332563456:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 161us (9284us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826296.444629:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11050 00000100:00100000:2.0:1787826296.444658:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11051 00000100:00100000:2.0:1787826296.444665:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:6166:x1874663332563584:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826296.444677:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826296.444690:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a23c lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801242 expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.445474:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f7961d81800) 00010000:00010000:2.0:1787826296.445496:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a1da lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080121f expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.445507:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:2.0:1787826296.445515:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f49f9c lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e46:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608011a8 expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826296.445523:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e46:0x0].0x0 (ffff9f7967ac1800) 00010000:00010000:2.0:1787826296.445528:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826296.445625:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:6166:x1874663332563584:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 958us (5829us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826296.445642:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11051 00010000:00010000:0.0:1787826297.007012:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826298.028439:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826298.073641:0:165467:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:3.0:1787826298.073693:0:165467:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:3.0:1787826299.052372:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826299.106066:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826299.106133:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826299.106169:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332564096 00000100:00100000:0.0:1787826299.106179:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332564096 00000100:00100000:0.0:1787826299.106239:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8790 00000100:00100000:0.0:1787826299.106249:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332564096:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826299.106326:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332564096:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 79us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826299.106332:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8790 00000100:00080000:0.0:1787826299.106353:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826299.106678:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332563840 00000100:00100000:3.0:1787826299.106684:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332563840 00000100:00100000:3.0:1787826299.106819:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14264 00000100:00100000:3.0:1787826299.106830:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332563840:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826299.106865:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332563840:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 39us (808us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.106871:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14264 00000100:00080000:0.0:1787826299.108190:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826299.115668:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.115703:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.115729:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.115764:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.115781:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.115797:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.115831:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.115859:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332564352 00000100:00100000:3.0:1787826299.115868:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332564352 00000100:00100000:3.0:1787826299.115923:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332563968 00000100:00100000:3.0:1787826299.115926:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332563968 00000100:00100000:3.0:1787826299.115946:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332564480 00000100:00100000:3.0:1787826299.115949:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332564480 00000100:00100000:3.0:1787826299.115972:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14265 00000100:00100000:3.0:1787826299.115983:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332564352:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826299.116075:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332564352:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 93us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.116082:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14265 00000100:00100000:3.0:1787826299.116098:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14266 00000100:00100000:3.0:1787826299.116123:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332563968:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826299.116150:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332563968:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 46us (386us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.116156:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14266 00000100:00100000:3.0:1787826299.116167:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14267 00000100:00100000:3.0:1787826299.116173:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332564480:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826299.116195:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332564480:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.116201:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14267 00000100:00100000:3.0:1787826299.116218:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332564608 00000100:00100000:3.0:1787826299.116221:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332564608 00000100:00100000:3.0:1787826299.116247:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332564224 00000100:00100000:3.0:1787826299.116249:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332564224 00000100:00100000:3.0:1787826299.116263:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332564736 00000100:00100000:3.0:1787826299.116265:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332564736 00000100:00100000:3.0:1787826299.116281:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8791 00000100:00100000:3.0:1787826299.116286:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332564608:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826299.116307:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332564608:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.116312:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8791 00000100:00100000:3.0:1787826299.116321:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8792 00000100:00100000:3.0:1787826299.116327:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332564224:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826299.116344:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332564224:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 17us (562us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.116353:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8792 00000100:00100000:3.0:1787826299.116362:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8793 00000100:00100000:3.0:1787826299.116367:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332564736:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826299.116386:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332564736:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 19us (555us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.116392:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8793 00000100:00100000:3.0:1787826299.116411:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332563712 00000100:00100000:3.0:1787826299.116415:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332563712 00000100:00100000:3.0:1787826299.116441:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2216 00000100:00100000:3.0:1787826299.116447:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf40050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332563712:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826299.116468:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf40050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332563712:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (738us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.116474:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2216 00000100:00080000:3.0:1787826299.116507:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826299.116511:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826299.116513:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826299.116515:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826299.618274:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826299.618370:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332564864 00000100:00100000:2.0:1787826299.618381:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332564864 00002000:00100000:2.0:1787826299.618435:0:8448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x51e:0x0] (0->5) req@ffff9f784b661f80 x1874663332564864/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:655/0 lens 496/0 e 0 to 0 dl 1787826310 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826299.618446:0:8448:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b661f80 with x1874663332564864 ext(0->5) 00002000:00010000:2.0:1787826299.618452:0:8448:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784b661f80 00000100:00100000:2.0:1787826299.618489:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27639 00000100:00100000:2.0:1787826299.618507:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332564864:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826299.618839:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826299.619308:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332564992 00000100:00100000:0.0:1787826299.619315:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332564992 00000100:00100000:0.0:1787826299.619360:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30805 00000100:00100000:0.0:1787826299.619369:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332564992:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:2.0:1787826299.621164:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332564864:12345-192.168.203.56@tcp:4:bash.0 Request processed in 2654us (2887us total) trans 4294973842 rc 0/0 00000100:00100000:2.0:1787826299.621182:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27639 00002000:00100000:2.0:1787826299.621196:0:8448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x51e:0x0] (0->5) req@ffff9f784b661f80 x1874663332564864/t4294973842(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:655/0 lens 496/448 e 0 to 0 dl 1787826310 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826299.621206:0:8448:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b661f80 with x1874663332564864 ext(0->5) 00002000:00010000:2.0:1787826299.621213:0:8448:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784b661f80 00000001:00080000:2.0:1787826299.625396:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973842 is committed 00000100:00100000:0.0:1787826299.625600:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332564992:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 6225us (6755us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826299.625612:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30805 00000100:00100000:3.0:1787826299.634338:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.634452:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826299.635065:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332565248 00000100:00100000:0.0:1787826299.635075:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332565248 00000100:00100000:0.0:1787826299.635159:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30806 00000100:00100000:0.0:1787826299.635170:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332565248:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:0.0:1787826299.635277:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332565248:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 108us (824us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826299.635284:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30806 00000100:00100000:2.0:1787826299.635817:0:20151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332565120 00000100:00100000:2.0:1787826299.635824:0:20151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332565120 00002000:00100000:2.0:1787826299.635856:0:20151:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x4c9:0x0] (0->5) req@ffff9f7849dd2680 x1874663332565120/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:655/0 lens 496/0 e 0 to 0 dl 1787826310 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826299.635867:0:20151:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd2680 with x1874663332565120 ext(0->5) 00002000:00010000:2.0:1787826299.635873:0:20151:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7849dd2680 00000100:00100000:2.0:1787826299.635900:0:20151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27640 00000100:00100000:2.0:1787826299.635909:0:20151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332565120:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826299.637809:0:20151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332565120:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1890us (3470us total) trans 4294976152 rc 0/0 00000100:00100000:2.0:1787826299.637822:0:20151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27640 00002000:00100000:2.0:1787826299.637834:0:20151:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x4c9:0x0] (0->5) req@ffff9f7849dd2680 x1874663332565120/t4294976152(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:655/0 lens 496/448 e 0 to 0 dl 1787826310 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826299.637843:0:20151:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd2680 with x1874663332565120 ext(0->5) 00002000:00010000:2.0:1787826299.637848:0:20151:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7849dd2680 00000100:00100000:2.0:1787826299.886904:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346168576:0@lo:400:kworker.0 00000100:00100000:2.0:1787826299.887017:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826299.887034:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346168832:0@lo:400:kworker.0 00000100:00100000:2.0:1787826299.887075:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346169088:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.887082:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826299.887144:0:30904:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346168576 00000100:00100000:2.0:1787826299.887152:0:30904:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346168576 00000100:00100000:3.0:1787826299.887160:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346169344:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.887190:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826299.887208:0:30904:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2217 00000100:00100000:2.0:1787826299.887220:0:30904:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf43450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346168576:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.887230:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346169856:0@lo:400:kworker.0 00000100:00100000:1.0:1787826299.887546:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346168704:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.890774:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826299.890857:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1787826299.890870:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346168960:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.890919:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826299.890944:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346169472:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.890966:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826299.890982:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346169984:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.891016:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826299.891062:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346168832 00000100:00100000:3.0:1787826299.891070:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346168832 00000100:00100000:3.0:1787826299.891150:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346169344 00000100:00100000:3.0:1787826299.891154:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346169344 00000100:00100000:3.0:1787826299.891181:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346168960 00000100:00100000:3.0:1787826299.891183:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346168960 00000100:00100000:3.0:1787826299.891200:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346169472 00000100:00100000:3.0:1787826299.891203:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346169472 00000100:00100000:3.0:1787826299.891236:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14268 00000100:00100000:3.0:1787826299.891487:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346168832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.891554:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346168832:12345-0@lo:400:kworker.0 Request processed in 309us (4475us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.891563:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14268 00000100:00100000:3.0:1787826299.891584:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14269 00000100:00100000:3.0:1787826299.891591:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346169344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.891698:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346169344:12345-0@lo:400:kworker.0 Request processed in 107us (4507us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.891705:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14269 00000100:00100000:3.0:1787826299.891719:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14270 00000100:00100000:3.0:1787826299.891736:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346168960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.891767:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346168960:12345-0@lo:400:kworker.0 Request processed in 42us (850us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.891772:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14270 00000100:00100000:3.0:1787826299.891784:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14271 00000100:00100000:3.0:1787826299.891790:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346169472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.891815:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346169472:12345-0@lo:400:kworker.0 Request processed in 27us (849us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.891820:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14271 00000100:00100000:3.0:1787826299.891872:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346169856 00000100:00100000:3.0:1787826299.891877:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346169856 00000100:00100000:3.0:1787826299.891906:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346169984 00000100:00100000:3.0:1787826299.891908:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346169984 00000100:00100000:3.0:1787826299.891929:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8794 00000100:00100000:3.0:1787826299.891934:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346169856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.891969:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346169856:12345-0@lo:400:kworker.0 Request processed in 35us (1203us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.891975:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8794 00000100:00100000:3.0:1787826299.891987:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8795 00000100:00100000:3.0:1787826299.892002:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346169984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.892033:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346169984:12345-0@lo:400:kworker.0 Request processed in 41us (1016us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.892039:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8795 00000100:00080000:3.0:1787826299.892069:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826299.892074:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826299.892077:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826299.892079:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826299.892224:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826299.892312:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346169088 00000100:00100000:0.0:1787826299.892318:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346169088 00000100:00100000:0.0:1787826299.892358:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14272 00000100:00100000:0.0:1787826299.892365:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346169088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.892462:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346169600:0@lo:400:kworker.0 00000100:00100000:0.0:1787826299.892599:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346169088:12345-0@lo:400:kworker.0 Request processed in 233us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826299.892606:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14272 00000100:00100000:1.0:1787826299.892725:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826299.892960:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346169216:0@lo:400:kworker.0 00000100:00100000:1.0:1787826299.892989:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1787826299.893219:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826299.893542:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346169728:0@lo:400:kworker.0 00000100:00100000:1.0:1787826299.893572:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826299.893599:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826299.893606:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346170112:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.893700:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346169216 00000100:00100000:3.0:1787826299.893707:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346169216 00000100:00100000:3.0:1787826299.893761:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8796 00000100:00100000:3.0:1787826299.893771:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346169216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.893815:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346169216:12345-0@lo:400:kworker.0 Request processed in 46us (825us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.893821:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8796 00000100:00080000:3.0:1787826299.893842:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826299.893851:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826299.893918:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346169088:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.893944:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346169600 00000100:00100000:3.0:1787826299.893948:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346169600 00000100:00100000:3.0:1787826299.893978:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8797 00000100:00100000:3.0:1787826299.893985:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346169600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826299.894771:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826299.894818:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346169216:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.895547:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346169600:12345-0@lo:400:kworker.0 Request processed in 1560us (1692us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.895557:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8797 00000100:00100000:3.0:1787826299.895585:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346170112 00000100:00100000:3.0:1787826299.895590:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346170112 00000100:00100000:3.0:1787826299.895626:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14273 00000100:00100000:3.0:1787826299.895633:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346170112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.895672:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346170112:12345-0@lo:400:kworker.0 Request processed in 41us (904us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.895678:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14273 00000100:00080000:3.0:1787826299.895696:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826299.895717:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346170112:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.895760:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346168960:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.895772:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346169472:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.895785:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346169984:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.895797:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346168832:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.895808:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346169344:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.895819:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346169856:0@lo:400:kworker.0 00000100:00100000:1.0:1787826299.895936:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346169600:0@lo:400:kworker.0 00000100:00100000:0.0:1787826299.896085:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346168704 00000100:00100000:0.0:1787826299.896090:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346168704 00000100:00100000:0.0:1787826299.896141:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346169728 00000100:00100000:0.0:1787826299.896144:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346169728 00000100:00100000:0.0:1787826299.896170:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23501 00000100:00100000:0.0:1787826299.896179:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346168704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.896197:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23502 00000100:00100000:3.0:1787826299.896208:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346169728:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826299.896233:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346168704:12345-0@lo:400:kworker.0 Request processed in 55us (3508us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826299.896240:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23501 00000100:00100000:0.0:1787826299.896280:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346168704:0@lo:400:kworker.0 00000100:00100000:3.0:1787826299.896345:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346169728:12345-0@lo:400:kworker.0 Request processed in 139us (2771us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.896352:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23502 00000100:00100000:0.0:1787826299.897059:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346169728:0@lo:400:kworker.0 00010000:00010000:0.0:1787826299.898891:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a0d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826299.898911:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a0d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826299.899136:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a0d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826299.899144:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826299.899169:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a0d0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a0d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826299.899192:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985369 is committed 00000020:00080000:0.0:1787826299.899210:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7851057c00 refcount 2 committed 0 result 0 batchid 3405 00000020:00080000:0.0:1787826299.899213:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1b500 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7851057180 00000020:00080000:0.0:1787826299.899216:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1508 00000020:00080000:0.0:1787826299.899218:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1b800 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e774e00 00000020:00080000:0.0:1787826299.899221:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1190 00000001:00080000:0.0:1787826299.899239:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985366 is committed 00000020:00080000:0.0:1787826299.899244:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7852f25c00 refcount 2 committed 0 result 0 batchid 3404 00000020:00080000:0.0:1787826299.899247:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852f25180 00000020:00080000:0.0:1787826299.899249:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1507 00000020:00080000:0.0:1787826299.899251:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e775400 00000020:00080000:0.0:1787826299.899253:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1188 00000004:00080000:0.0:1787826299.899259:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 363 committed 00000020:00080000:0.0:1787826299.899268:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7852f25540 refcount 2 committed 0 result 0 batchid 3403 00000020:00080000:0.0:1787826299.899271:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852f25cc0 00000020:00080000:0.0:1787826299.899273:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1506 00000020:00080000:0.0:1787826299.899274:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839d00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e774a00 00000020:00080000:0.0:1787826299.899276:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1187 00000020:00080000:0.0:1787826299.899282:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7852405a80 refcount 2 committed 0 result 0 batchid 3402 00000020:00080000:0.0:1787826299.899284:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1be00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852405600 00000020:00080000:0.0:1787826299.899287:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1505 00000020:00080000:0.0:1787826299.899288:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1b700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78472a3a00 00000020:00080000:0.0:1787826299.899291:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1186 00000001:00080000:0.0:1787826299.899297:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985360 is committed 00000020:00080000:0.0:1787826299.899308:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3401 batchid updated 00000100:00100000:0.0:1787826299.899327:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826299.899345:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346170240:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826299.899417:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826299.899459:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346170240 00000100:00100000:0.0:1787826299.899466:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346170240 00000100:00100000:0.0:1787826299.899511:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11052 00000100:00100000:0.0:1787826299.899520:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd83850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346170240:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787826299.899535:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826299.899556:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a0d7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x18f:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a0d0 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826299.899571:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x18f:0x0].0x5a734ce9 (ffff9f7967ac1a00) 00010000:00010000:0.0:1787826299.899579:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826299.899612:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd83850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346170240:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 93us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826299.899619:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11052 00000100:00100000:0.0:1787826299.899694:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346170240:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826299.899759:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899770:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899776:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899783:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826299.899787:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899798:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899810:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:1.0:1787826299.899813:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826299.899817:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899823:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a18d lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899851:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899859:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899865:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899870:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826299.899873:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899878:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899886:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:1.0:1787826299.899888:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826299.899892:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899902:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a194 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.899918:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.901183:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.901192:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.901198:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826299.901201:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.901209:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.901358:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x4ec94fd6 (ffff9f7961d81900) 00010000:00010000:1.0:1787826299.901363:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826299.901367:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.901375:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a235 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826299.901898:0:30904:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf43450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346168576:12345-0@lo:400:kworker.0 Request processed in 14675us (14875us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826299.901911:0:30904:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2217 00000020:00080000:2.0:1787826299.902139:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3405 committed batchid 3401 00000004:00080000:2.0:1787826299.902254:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784c4f43c0 with version 6808 00000020:00080000:2.0:1787826299.902299:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3398 cancel update log [0x200000401:0x3:0x0].1501: rc = 0 00000004:00080000:1.0:1787826299.902331:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6808 rpc_version 6808 00000020:00080000:2.0:1787826299.902346:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3398 cancel update log [0x240000400:0x3:0x0].1182: rc = 0 00000020:00100000:1.0:1787826299.902365:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000004:00080000:2.0:1787826299.902376:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784c4f4cc0 with version 6809 00000020:00080000:2.0:1787826299.902381:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3399 cancel update log [0x200000401:0x3:0x0].1502: rc = 0 00000020:00080000:2.0:1787826299.902403:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3399 cancel update log [0x240000400:0x3:0x0].1183: rc = 0 00000004:00080000:2.0:1787826299.902422:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784c4f4e40 with version 6810 00000020:00080000:2.0:1787826299.902425:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3400 cancel update log [0x200000401:0x3:0x0].1503: rc = 0 00000020:00080000:2.0:1787826299.902442:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3400 cancel update log [0x240000400:0x3:0x0].1184: rc = 0 00000004:00080000:2.0:1787826299.902460:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784c4f4180 with version 6811 00000020:00080000:2.0:1787826299.902462:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3401 cancel update log [0x200000401:0x3:0x0].1504: rc = 0 00000020:00080000:2.0:1787826299.902484:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3401 cancel update log [0x240000400:0x3:0x0].1185: rc = 0 00000020:00080000:2.0:1787826299.902493:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3401 00000100:00100000:2.0:1787826299.902571:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346170624:0@lo:6:osp-syn-0-1.0 00000100:00100000:2.0:1787826299.902627:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826299.902674:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346170624 00000100:00100000:2.0:1787826299.902681:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346170624 00000100:00100000:2.0:1787826299.902716:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30807 00000100:00100000:2.0:1787826299.902723:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346170624:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:2.0:1787826299.902759:0:35097:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1227 count 1 00010000:00010000:2.0:1787826299.902782:0:35097:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826299.902802:0:35097:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 6 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826299.902811:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4cb:0x0].0x0 since=10s v=(0 0) 00010000:00010000:2.0:1787826299.902816:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826299.902820:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826299.902843:0:35097:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 6 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826299.902857:0:35097:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 6 pid: 35352 timeout: 8212 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826299.902869:0:35097:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:35097:1874663346170752:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826299.902929:0:35097:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826299.903619:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346168576:0@lo:400:kworker.0 00000100:00100000:1.0:1787826299.903773:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784c6bb100 x1874663346170496/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826299.903800:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346170496:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826299.903854:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826299.903884:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826299.910025:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346170496 00000100:00100000:1.0:1787826299.910042:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346170496 00000100:00100000:1.0:1787826299.910139:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23503 00000100:00100000:1.0:1787826299.910150:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346170496:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826299.910530:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346170496:12345-0@lo:1000:osp_up0-1.0 Request processed in 381us (6674us total) trans 8589957356 rc 0/0 00000100:00100000:1.0:1787826299.910538:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23503 00000100:00100000:1.0:1787826299.910597:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346170496:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826299.910611:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6809 rpc_version 6809 00000020:00100000:1.0:1787826299.910634:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826299.910641:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b758000 x1874663346171008/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826299.910662:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b758000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346171008:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826299.910699:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826299.910722:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826299.910748:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346171008 00000100:00100000:1.0:1787826299.910752:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346171008 00000100:00100000:1.0:1787826299.910780:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23504 00000100:00100000:1.0:1787826299.910787:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346171008:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826299.910874:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346171008:12345-0@lo:1000:osp_up0-1.0 Request processed in 87us (174us total) trans 8589957357 rc 0/0 00000100:00100000:1.0:1787826299.910880:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23504 00000100:00100000:1.0:1787826299.910914:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b758000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346171008:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826299.910920:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6810 rpc_version 6810 00000020:00100000:1.0:1787826299.910932:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826299.910938:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b759c00 x1874663346171264/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826299.910950:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346171264:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826299.910975:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826299.910990:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851057480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826299.911001:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346171264 00000100:00100000:1.0:1787826299.911004:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346171264 00000100:00100000:1.0:1787826299.911025:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23505 00000100:00100000:1.0:1787826299.911031:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346171264:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826299.911099:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346171264:12345-0@lo:1000:osp_up0-1.0 Request processed in 67us (122us total) trans 8589957358 rc 0/0 00000100:00100000:1.0:1787826299.911127:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23505 00000100:00100000:1.0:1787826299.911160:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346171264:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826299.911167:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6811 rpc_version 6811 00000020:00100000:1.0:1787826299.911178:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826299.911183:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b759880 x1874663346171520/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826299.911196:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346171520:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826299.911222:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826299.911235:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f25a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826299.912150:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346171520 00000100:00100000:1.0:1787826299.912159:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346171520 00000100:00100000:1.0:1787826299.912205:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23506 00000100:00100000:1.0:1787826299.912214:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346171520:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826299.912612:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346171520:12345-0@lo:1000:osp_up0-1.0 Request processed in 397us (1388us total) trans 8589957359 rc 0/0 00000100:00100000:1.0:1787826299.912621:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23506 00000100:00100000:3.0:1787826299.912775:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346171520:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826299.914015:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826299.914050:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332565376 00000100:00100000:3.0:1787826299.914058:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332565376 00010000:00010000:3.0:1787826299.914129:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 7 pid: 35352 timeout: 8212 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826299.914166:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21903 00000100:00100000:3.0:1787826299.914177:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332565376:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826299.914193:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826299.914199:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 7 pid: 35352 timeout: 8212 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.914214:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 7 pid: 35352 timeout: 8212 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.914242:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826299.914247:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826299.914254:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.914264:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.914285:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826299.914341:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332565376:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 166us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826299.914348:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21903 00010000:00010000:1.0:1787826299.914822:0:35097:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f49ef4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801169 expref: 6 pid: 35352 timeout: 8212 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826299.914838:0:35097:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:35097:1874663346170752:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826299.914865:0:35097:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.914877:0:35097:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.914885:0:35097:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.914895:0:35097:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.914904:0:35097:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826299.914929:0:35097:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826299.915165:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346170624:12345-0@lo:6:osp-syn-0-1.0 Request processed in 12440us (12536us total) trans 4294973843 rc 0/0 00000100:00100000:1.0:1787826299.915178:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30807 00000004:00080000:1.0:1787826299.915228:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784836e300/2, rc 0, transno 4294973843 00000100:00100000:1.0:1787826299.915235:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346170624:0@lo:6:osp-syn-0-1.0 00010000:00010000:3.0:1787826299.915659:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826299.915668:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.915676:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826299.915681:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.915694:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.915711:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.915718:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aef24583/0xe21309aa81f4a251 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826299.915753:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826299.915755:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826299.915757:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826300.076403:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826300.396373:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787826300.396397:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346171648:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826300.396491:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826300.396545:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346171648 00000100:00100000:2.0:1787826300.396552:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346171648 00000100:00100000:2.0:1787826300.396602:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6215 00000100:00100000:2.0:1787826300.396611:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346171648:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1787826300.396635:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826300.396830:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346171648:12345-0@lo:13:osp-pre-1-1.0 Request processed in 219us (338us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826300.396837:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6215 00000100:00100000:3.0:1787826300.396937:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346171648:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826300.482577:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826300.482683:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332565504 00000100:00100000:3.0:1787826300.482694:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332565504 00000100:00100000:3.0:1787826300.482795:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68338 00000100:00100000:3.0:1787826300.482807:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224455:x1874663332565504:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826300.482829:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826300.482852:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a258 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801250 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.482923:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.483126:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.483189:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826300.483289:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826300.483293:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826300.483320:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.483353:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a258 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.483431:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.483442:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826300.483445:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826300.483449:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826300.483515:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224455:x1874663332565504:12345-192.168.203.56@tcp:101:cat.0 Request processed in 710us (949us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.483523:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68338 00000100:00100000:1.0:1787826300.489493:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826300.489667:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332565632 00000100:00100000:3.0:1787826300.489677:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332565632 00000100:00100000:3.0:1787826300.489753:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68339 00000100:00100000:3.0:1787826300.489765:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843436d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224455:x1874663332565632:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826300.489788:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826300.489810:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a266 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801257 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.489884:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.489898:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.489908:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.489923:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.489929:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.489935:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490044:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a27b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490052:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a27b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490058:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a27b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826300.490078:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a27b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490093:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490100:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490133:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490140:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490145:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826300.490149:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490156:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490168:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826300.490171:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826300.490176:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490181:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a26d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490194:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490199:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490204:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490210:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826300.490213:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490220:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490228:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79460a2200) 00010000:00010000:3.0:1787826300.490230:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826300.490233:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490239:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a274 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490275:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a266 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801257 expref: 9 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490290:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a27b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801257 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.490298:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2800) 00010000:00010000:3.0:1787826300.490300:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826300.490303:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826300.490370:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843436d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224455:x1874663332565632:12345-192.168.203.56@tcp:101:cat.0 Request processed in 607us (886us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.490379:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68339 00000100:00100000:1.0:1787826300.493757:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826300.493831:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332565760 00000100:00100000:3.0:1787826300.493842:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332565760 00000100:00100000:3.0:1787826300.493907:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68340 00000100:00100000:3.0:1787826300.493919:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224455:x1874663332565760:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826300.493941:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826300.493965:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a282 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080125e expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.496588:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4a289 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.496615:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4a289 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.496630:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4a289 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826300.496752:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e48:0x0]: returning size 4096 00000004:00200000:3.0:1787826300.496758:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826300.496813:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a282 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080125e expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.496836:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4a289 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080125e expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.496845:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826300.496849:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826300.496853:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826300.496932:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224455:x1874663332565760:12345-192.168.203.56@tcp:101:cat.0 Request processed in 3013us (3181us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.496944:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68340 00000100:00100000:1.0:1787826300.509160:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826300.509514:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332565888 00000100:00100000:0.0:1787826300.509524:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332565888 00000100:00100000:0.0:1787826300.509584:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68341 00000100:00100000:0.0:1787826300.509596:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224455:x1874663332565888:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826300.509617:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826300.509643:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4a290 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801265 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.509701:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.509715:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.509736:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826300.509782:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x192:0x0]: returning size 4096 00000004:00200000:0.0:1787826300.509787:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826300.509820:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4a290 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.509837:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.509847:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f794b57ac00) 00010000:00010000:0.0:1787826300.509850:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826300.509853:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:0.0:1787826300.509916:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224455:x1874663332565888:12345-192.168.203.56@tcp:101:cat.0 Request processed in 323us (763us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826300.509924:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68341 00000100:00100000:1.0:1787826300.517330:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826300.517410:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332566016 00000100:00100000:3.0:1787826300.517423:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332566016 00000100:00100000:3.0:1787826300.517489:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68342 00000100:00100000:3.0:1787826300.517502:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224455:x1874663332566016:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826300.517525:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826300.517599:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a29e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080126c expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.517676:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.517689:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.517699:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826300.518442:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x190:0x0]: returning size 4096 00000004:00200000:3.0:1787826300.518450:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826300.518499:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a29e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.518516:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.518524:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2200) 00010000:00010000:3.0:1787826300.518527:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826300.518531:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826300.518599:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224455:x1874663332566016:12345-192.168.203.56@tcp:101:cat.0 Request processed in 1100us (1279us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.518608:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68342 00000100:00100000:3.0:1787826300.523222:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826300.525006:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332566272 00000100:00100000:0.0:1787826300.525021:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332566272 00000100:00100000:0.0:1787826300.525124:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68343 00000100:00100000:0.0:1787826300.525138:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224455:x1874663332566272:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826300.525161:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826300.525184:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a2ac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801273 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.525265:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.525278:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.525289:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.525300:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 3/1,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.525307:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826300.525313:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:0.0:1787826300.525399:0:35098:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000100:00100000:3.0:1787826300.908363:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346171776:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826300.908436:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826300.908497:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826300.908505:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346171904:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826300.908545:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826300.908624:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346172032:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826300.908658:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826300.908692:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346171776 00000100:00100000:3.0:1787826300.908699:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346171776 00000100:00100000:3.0:1787826300.908755:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346172032 00000100:00100000:3.0:1787826300.908759:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346172032 00000100:00100000:3.0:1787826300.908780:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23507 00000100:00100000:3.0:1787826300.908789:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346171776:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787826300.908833:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442807808 avail 1296568320 00000100:00100000:3.0:1787826300.908879:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346171776:12345-0@lo:41:osp-pre-0-1.0 Request processed in 92us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.908886:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23507 00000100:00100000:3.0:1787826300.908922:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23508 00000100:00100000:3.0:1787826300.908931:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346172032:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787826300.908947:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443422208 avail 1297182720 00000100:00100000:3.0:1787826300.908978:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346172032:12345-0@lo:41:osp-pre-1-0.0 Request processed in 48us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.908983:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23508 00000100:00100000:3.0:1787826300.909028:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985360 gen 2 00000004:00080000:3.0:1787826300.909034:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985359 committed transno 4294985360 00000020:00080000:3.0:1787826300.909040:0:3654:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f784549f000 refcount 2 committed 0 result 0 batchid 4294979079 00000020:00080000:3.0:1787826300.909045:0:3654:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1b100 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784549f540 00000020:00080000:3.0:1787826300.909048:0:3654:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].935 00000020:00080000:3.0:1787826300.909055:0:3654:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797c2e2a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78472a3800 00000020:00080000:3.0:1787826300.909058:0:3654:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].932 00000004:00080000:3.0:1787826300.909081:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985360 committed transno 4294985360 00000100:00100000:3.0:1787826300.909140:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346172032:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826300.909171:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346171776:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826300.909198:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346171904 00000100:00100000:3.0:1787826300.909203:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346171904 00000100:00100000:3.0:1787826300.909231:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6216 00000100:00100000:3.0:1787826300.909238:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346171904:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826300.909256:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662454784 00000100:00100000:3.0:1787826300.909305:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346171904:12345-0@lo:13:osp-pre-0-1.0 Request processed in 67us (759us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.909310:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6216 00000100:00100000:3.0:1787826300.909341:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346171904:0@lo:13:osp-pre-0-1.0 00000020:00080000:2.0:1787826300.913788:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979079 committed batchid 4294979078 00000020:00080000:2.0:1787826300.913911:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979078 00000100:00100000:0.0:1787826300.914093:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826300.914136:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346172160:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826300.914208:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826300.914296:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346172160 00000100:00100000:2.0:1787826300.914305:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346172160 00000100:00100000:2.0:1787826300.914356:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6217 00000100:00100000:2.0:1787826300.914366:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346172160:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826300.914393:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914711040 avail 3683188736 00000100:00100000:2.0:1787826300.914460:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346172160:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826300.914467:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6217 00000100:00100000:0.0:1787826300.914912:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346172160:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826300.915590:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826300.915605:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346172288:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826300.915657:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826300.915783:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346172288 00000100:00100000:3.0:1787826300.915790:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346172288 00000100:00100000:3.0:1787826300.915837:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6218 00000100:00100000:3.0:1787826300.915850:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346172288:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787826300.915870:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662454784 00000100:00100000:3.0:1787826300.915917:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346172288:12345-0@lo:13:osp-pre-0-0.0 Request processed in 68us (259us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.915927:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6218 00000100:00100000:1.0:1787826300.916059:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346172288:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1787826300.917334:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976152 is committed 00000001:00080000:1.0:1787826300.917350:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976151 is committed 00000100:00100000:1.0:1787826300.997781:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826300.998146:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332566400 00000100:00100000:3.0:1787826300.998157:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332566400 00000100:00100000:3.0:1787826300.998605:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68344 00000100:00100000:3.0:1787826300.998620:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224457:x1874663332566400:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826300.998642:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826300.998667:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a2c1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080127a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.998746:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.998760:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.998773:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826300.998820:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826300.998825:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826300.998845:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.998871:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a2c1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.998891:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826300.998901:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826300.998904:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826300.998908:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826300.998984:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224457:x1874663332566400:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 368us (1212us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826300.999000:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68344 00000100:00100000:3.0:1787826301.004647:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826301.004772:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332566528 00000100:00100000:2.0:1787826301.004785:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332566528 00000100:00100000:2.0:1787826301.004875:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68345 00000100:00100000:2.0:1787826301.004888:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224457:x1874663332566528:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826301.004916:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826301.004944:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4a2cf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801281 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826301.005619:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787826301.005738:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.005756:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.005769:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.006057:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.006078:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.006089:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007680:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007703:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007714:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826301.007762:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007779:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007786:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007793:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007800:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007806:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826301.007811:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007819:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007833:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826301.007836:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826301.007841:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007847:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4a2d6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007866:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007873:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007879:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007884:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826301.007887:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007893:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007899:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9b00) 00010000:00010000:2.0:1787826301.007902:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826301.007904:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007910:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a2dd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007951:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4a2cf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007967:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.007974:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2800) 00010000:00010000:2.0:1787826301.007977:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826301.007980:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826301.008068:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224457:x1874663332566528:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 3181us (3434us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.008077:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68345 00000100:00100000:1.0:1787826301.013388:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826301.013524:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332566656 00000100:00100000:3.0:1787826301.013538:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332566656 00000100:00080000:0.0:1787826301.013619:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826301.013631:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68346 00000100:00100000:3.0:1787826301.013649:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:224457:x1874663332566656:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826301.013675:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826301.013704:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a2eb lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801288 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.013800:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a2f2 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.013816:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a2f2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.013833:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a2f2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826301.013951:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e48:0x0]: returning size 4096 00000004:00200000:3.0:1787826301.013957:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826301.014002:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a2eb lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801288 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.014213:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a2f2 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801288 expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.014224:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826301.014227:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826301.014231:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826301.014488:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224457:x1874663332566656:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 844us (1114us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.014497:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68346 00000100:00100000:1.0:1787826301.018931:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826301.025647:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332566784 00000100:00100000:0.0:1787826301.025668:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332566784 00000100:00100000:0.0:1787826301.027489:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68347 00000100:00100000:0.0:1787826301.027516:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224457:x1874663332566784:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826301.027546:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826301.027571:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a2f9 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080128f expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.027654:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.027670:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.027683:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.033315:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.033339:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.033352:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.033480:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a30e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.033488:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a30e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.033495:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a30e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826301.033551:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e49:0x0]: returning size 6 00000004:00200000:0.0:1787826301.033556:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826301.034383:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a30e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034404:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a30e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034415:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034422:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034428:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034434:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034440:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826301.034444:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034451:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034465:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f794b57ac00) 00010000:00010000:0.0:1787826301.034468:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826301.034473:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034483:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a300 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034498:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034505:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034510:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034514:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826301.034516:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034522:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034529:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x4ec94fd6 (ffff9f794b57ad00) 00010000:00010000:0.0:1787826301.034531:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826301.034534:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.034539:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a307 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.035485:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a2f9 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080128f expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.035514:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a30e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080128f expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.035522:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:0.0:1787826301.035525:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826301.035527:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826301.036745:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224457:x1874663332566784:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 9230us (17817us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.036758:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68347 00000100:00100000:1.0:1787826301.045400:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826301.045549:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332566912 00000100:00100000:3.0:1787826301.045559:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332566912 00000100:00100000:3.0:1787826301.045705:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68348 00000100:00100000:3.0:1787826301.045717:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224457:x1874663332566912:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826301.045754:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826301.045776:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a315 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801296 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.045843:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.045858:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.045870:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.045933:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.045942:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.045949:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046065:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4a32a lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046075:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4a32a lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046082:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4a32a lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826301.046144:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db8:0x0]: returning size 6 00000004:00200000:3.0:1787826301.046149:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826301.046167:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4a32a lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046179:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4a32a lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046191:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046197:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046207:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046213:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046219:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826301.046224:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046231:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046243:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2200) 00010000:00010000:3.0:1787826301.046245:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826301.046250:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 3/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046257:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a31c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046271:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046278:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046283:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046288:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826301.046291:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046296:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046303:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f7961d81200) 00010000:00010000:3.0:1787826301.046305:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826301.046308:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046314:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a323 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046340:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a315 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801296 expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046354:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4a32a lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801296 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.046362:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db8:0x0].0x0 (ffff9f79460a2600) 00010000:00010000:3.0:1787826301.046363:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826301.046366:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826301.046475:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224457:x1874663332566912:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 760us (1085us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.046483:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68348 00000100:00100000:1.0:1787826301.050511:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826301.050861:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826301.050970:0:125380:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332567296 00000100:00100000:0.0:1787826301.050981:0:125380:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332567296 00000100:00100000:0.0:1787826301.051188:0:125380:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1227 00000100:00100000:0.0:1787826301.051200:0:125380:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224457:x1874663332567296:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:0.0:1787826301.051252:0:125380:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826301.051279:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4a32a lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801296 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.051301:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db8:0x0].0x0 (ffff9f79460a2600) 00010000:00010000:0.0:1787826301.051311:0:125380:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826301.051401:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172416:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.051467:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.051495:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.051514:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346172416 00000100:00100000:0.0:1787826301.051529:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346172416 00000100:00100000:0.0:1787826301.051562:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23509 00000100:00100000:0.0:1787826301.051916:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172416:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.052008:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172416:12345-0@lo:1000:mdt_io00_003.0 Request processed in 438us (539us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.052028:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23509 00000100:00100000:0.0:1787826301.052088:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172416:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.052357:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172544:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.052388:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.052407:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826301.052755:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332567040 00000100:00100000:3.0:1787826301.052761:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332567040 00000100:00100000:3.0:1787826301.052804:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346172544 00000100:00100000:3.0:1787826301.052808:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346172544 00000100:00100000:3.0:1787826301.052852:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23510 00000100:00100000:3.0:1787826301.052860:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172544:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826301.052945:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172544:12345-0@lo:1000:mdt_io00_003.0 Request processed in 75us (546us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.052953:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23510 00000100:00100000:2.0:1787826301.053272:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11053 00000100:00100000:2.0:1787826301.053282:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332567040:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:2.0:1787826301.053293:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826301.053315:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a30e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080128f expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.053332:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f794b57a700) 00010000:00010000:2.0:1787826301.053344:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826301.053389:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332567040:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 109us (2889us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.053396:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11053 00000100:00100000:0.0:1787826301.053891:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172544:0@lo:1000:mdt_io00_003.0 00000004:00080000:0.0:1787826301.053929:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x192:0x0] 00000100:00100000:0.0:1787826301.053986:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172672:0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.054036:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.054057:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.054334:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346172672 00000100:00100000:2.0:1787826301.054341:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346172672 00000100:00100000:2.0:1787826301.054384:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23511 00000100:00100000:2.0:1787826301.054392:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172672:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826301.054539:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172672:12345-0@lo:1000:mdt_io00_003.0 Request processed in 147us (501us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.054546:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23511 00000100:00100000:0.0:1787826301.055375:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172672:0@lo:1000:mdt_io00_003.0 00000004:00080000:2.0:1787826301.055598:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e48:0x0] 00000100:00100000:2.0:1787826301.055805:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172800:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826301.055856:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826301.055933:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826301.056089:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346172800 00000100:00100000:1.0:1787826301.056096:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346172800 00000100:00100000:1.0:1787826301.056400:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23512 00000100:00100000:1.0:1787826301.056411:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172800:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826301.056630:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172800:0@lo:1000:mdt_io00_003.0 00000004:00080000:2.0:1787826301.056675:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x192:0x0] 00000100:00100000:2.0:1787826301.056700:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172928:0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826301.056748:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.056762:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172800:12345-0@lo:1000:mdt_io00_003.0 Request processed in 108us (660us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826301.056770:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23512 00000100:00100000:2.0:1787826301.056844:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826301.056914:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346172928 00000100:00100000:1.0:1787826301.056919:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346172928 00000100:00100000:1.0:1787826301.056957:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23513 00000100:00100000:1.0:1787826301.056964:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172928:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.057029:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346172928:12345-0@lo:1000:mdt_io00_003.0 Request processed in 64us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826301.057035:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23513 00000100:00100000:2.0:1787826301.057284:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346172928:0@lo:1000:mdt_io00_003.0 00000004:00020000:2.0:1787826301.057315:0:125380:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:2.0:1787826301.057360:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.057371:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.057381:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826301.057394:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f685500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346173056:0@lo:101:mdt_io00_003.0 00000100:00100000:2.0:1787826301.057420:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826301.057436:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4600 going to sleep for 11 seconds 00000004:00020000:1.0:1787826301.057502:0:35098:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4467 00010000:00010000:1.0:1787826301.057570:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057584:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057595:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826301.057608:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db8:0x0]: returning size 6 00010000:00010000:1.0:1787826301.057689:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057697:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057704:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057711:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057717:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826301.057732:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057740:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057753:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2200) 00010000:00010000:1.0:1787826301.057756:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826301.057760:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057766:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a2b3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057781:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057787:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057793:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057799:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826301.057802:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057808:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057816:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f7961d81200) 00010000:00010000:1.0:1787826301.057818:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826301.057821:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 2/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057827:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a2ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057871:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a2ac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057886:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.057894:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db8:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:1.0:1787826301.057897:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826301.057899:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826301.057962:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224455:x1874663332566272:12345-192.168.203.56@tcp:101:cat.0 Request processed in 532827us (534749us total) trans 8589957360 rc 0/0 00000100:00100000:1.0:1787826301.057970:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68343 00000100:00100000:1.0:1787826301.058000:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346173056 00000100:00100000:1.0:1787826301.058004:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346173056 00000100:00100000:1.0:1787826301.058226:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23514 00000100:00100000:1.0:1787826301.058235:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:125380:x1874663346173056:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:1.0:1787826301.058245:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826301.058260:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a331 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.058279:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.058288:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 4/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a331 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.058311:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.058325:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 6 pid: 8752 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826301.058336:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346173184:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826301.058377:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826301.062810:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826301.062892:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826301.062966:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332567552 00000100:00100000:3.0:1787826301.062975:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332567552 00000100:00100000:3.0:1787826301.063052:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30808 00000100:00100000:3.0:1787826301.063064:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332567552:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826301.063082:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826301.063102:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a34d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080129d expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826301.063192:0:35096:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:35096:1874663346173312:192.168.203.56@tcp:106: 00000100:00100000:3.0:1787826301.063278:0:35096:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.063802:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346173184:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826301.063835:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a331 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.063846:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f794b57ac00) 00010000:00010000:2.0:1787826301.063850:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826301.063854:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a331 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.063864:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:2.0:1787826301.063914:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346173056:12345-0@lo:101:mdt_io00_003.0 Request processed in 5680us (6493us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.063922:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23514 00000100:00100000:3.0:1787826301.064026:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f685500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346173056:0@lo:101:mdt_io00_003.0 00010000:00010000:3.0:1787826301.064045:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.064055:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826301.064230:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1787826301.067021:0:35096:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000401:0x51e:0x0] updating lvb size: 0 -> 6 00002000:00010000:3.0:1787826301.067031:0:35096:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000401:0x51e:0x0] updating lvb mtime: 0 -> 1787826295 00002000:00010000:3.0:1787826301.067034:0:35096:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000401:0x51e:0x0] updating lvb atime: 0 -> 1787826295 00002000:00010000:3.0:1787826301.067036:0:35096:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000401:0x51e:0x0] updating lvb ctime: 0 -> 1787826295 00002000:00010000:3.0:1787826301.067039:0:35096:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000401:0x51e:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:3.0:1787826301.067062:0:35096:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:35096:1874663346173312:192.168.203.56@tcp:106: 00010000:00010000:3.0:1787826301.067354:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a34d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080129d expref: 6 pid: 35096 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.067391:0:35096:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a34d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080129d expref: 6 pid: 35096 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.067405:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:3.0:1787826301.067497:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332567552:12345-192.168.203.56@tcp:101:cat.0 Request processed in 4435us (4603us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826301.067506:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30808 00000100:00100000:0.0:1787826301.067853:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332567424 00000100:00100000:0.0:1787826301.067861:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332567424 00010000:00010000:0.0:1787826301.068211:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 7 pid: 8752 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826301.068254:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21904 00000100:00100000:0.0:1787826301.068264:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332567424:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826301.068277:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826301.068284:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 7 pid: 8752 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068301:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 7 pid: 8752 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068319:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a297 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801265 expref: 7 pid: 8752 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068334:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f794b57ac00) 00010000:00010000:0.0:1787826301.068341:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a331 expref: 5 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068353:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a331 expref: 5 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068362:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a331 expref: 5 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068402:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a331 expref: 5 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826301.068416:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001_UUID:125406:1874663346173440:0@lo:105: 00000100:00100000:0.0:1787826301.068470:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.068504:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346173440 00000100:00100000:0.0:1787826301.068508:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346173440 00000100:00100000:0.0:1787826301.068862:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7840 00000100:00100000:0.0:1787826301.068872:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:125406:x1874663346173440:12345-0@lo:105: 00010000:00010000:0.0:1787826301.068885:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068896:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068905:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068913:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068920:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.068930:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.069308:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef600) 00000100:00100000:0.0:1787826301.069350:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:125406:x1874663346173440:12345-0@lo:105: Request processed in 479us (878us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.069357:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7840 00000100:00100000:0.0:1787826301.069649:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.069686:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001_UUID:125406:1874663346173440:0@lo:105: 00010000:00010000:0.0:1787826301.069705:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826301.069753:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332567424:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1490us (6951us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.069760:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21904 00000100:00080000:3.0:1787826301.069834:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1787826301.069849:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.069859:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826301.069881:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x192:0x0] 00010000:00010000:3.0:1787826301.069922:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.069931:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.070631:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826301.070662:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346173568:0@lo:101:mdt_io00_003.0 00000100:00100000:3.0:1787826301.070714:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826301.070757:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.070823:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346173568 00000100:00100000:2.0:1787826301.070831:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346173568 00000100:00100000:2.0:1787826301.070880:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23515 00000100:00100000:2.0:1787826301.070890:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:125380:x1874663346173568:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:2.0:1787826301.070906:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826301.070927:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a35b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a354 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.071332:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a35b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a354 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.071347:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a35b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a354 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.071354:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x4ec94fd6 (ffff9f7952ca9200) 00010000:00010000:2.0:1787826301.071357:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826301.071360:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826301.071417:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346173568:12345-0@lo:101:mdt_io00_003.0 Request processed in 528us (702us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.071424:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23515 00000100:00100000:0.0:1787826301.071629:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd1500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346173568:0@lo:101:mdt_io00_003.0 00010000:00010000:0.0:1787826301.071646:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a35b expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.071657:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a35b expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826301.071679:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x192:0x0] 00010000:00010000:0.0:1787826301.071700:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.071716:0:125380:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.072900:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.073437:0:125380:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.073454:0:125380:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 10 pid: 24703 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826301.073469:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346173696:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826301.073557:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.081768:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826301.081967:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332567680 00000100:00100000:3.0:1787826301.081979:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332567680 00000100:00100000:3.0:1787826301.082054:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30809 00000100:00100000:3.0:1787826301.082065:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224455:x1874663332567680:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826301.082083:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826301.082101:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012a4 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.082163:0:8759:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 5 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.082172:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x51e:0x0].0x0 since=5s v=(0 0) 00010000:00010000:3.0:1787826301.082176:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826301.082181:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.082215:0:8759:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 5 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.082229:0:8759:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 5 pid: 20167 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826301.082261:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346173824:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826301.082336:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.085534:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826301.085630:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332567808 00000100:00100000:3.0:1787826301.085641:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332567808 00010000:00010000:3.0:1787826301.085706:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 11 pid: 24703 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826301.085760:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21905 00000100:00100000:3.0:1787826301.085771:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332567808:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826301.085786:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826301.085793:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 11 pid: 24703 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.085807:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 11 pid: 24703 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.085830:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2200) 00010000:00010000:3.0:1787826301.085836:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.085846:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.085854:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.085872:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826301.085920:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332567808:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 151us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.085928:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21905 00000100:00080000:1.0:1787826301.086585:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787826301.088713:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a2a5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080126c expref: 10 pid: 24703 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826301.088749:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346173696:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826301.088776:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.089470:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.089482:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.089506:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.089519:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.089529:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826301.089602:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346173952:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.089654:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.089695:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.089785:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346173824:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826301.089814:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.089833:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:2.0:1787826301.089888:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224455:x1874663332567680:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7825us (8129us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.089897:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30809 00000100:00100000:3.0:1787826301.091859:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346173952 00000100:00100000:3.0:1787826301.091876:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346173952 00000100:00100000:3.0:1787826301.091952:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23516 00000100:00100000:3.0:1787826301.091963:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346173952:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826301.092489:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346173952:12345-0@lo:1000:mdt_io00_003.0 Request processed in 526us (2831us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.092499:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23516 00000100:00100000:2.0:1787826301.095567:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826301.095730:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332567936 00000100:00100000:3.0:1787826301.095741:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332567936 00010000:00010000:3.0:1787826301.095802:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 6 pid: 20167 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826301.095843:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21906 00000100:00100000:3.0:1787826301.095854:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332567936:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826301.095869:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826301.095892:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 6 pid: 20167 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.095906:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 6 pid: 20167 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.095932:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad45511c/0xe21309aa81f4a220 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080123b expref: 6 pid: 20167 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826301.096026:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787826301.096294:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826301.096299:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826301.096309:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.096320:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826301.096359:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826301.096538:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6511:1874663346174080:192.168.203.56@tcp:105: 00000100:00100000:3.0:1787826301.096624:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826301.098472:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346173952:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.098568:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174208:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.098630:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.098657:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.098983:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346174208 00000100:00100000:2.0:1787826301.099001:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346174208 00000100:00100000:2.0:1787826301.099057:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23517 00000100:00100000:2.0:1787826301.099067:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174208:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826301.099459:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174208:12345-0@lo:1000:mdt_io00_003.0 Request processed in 392us (827us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.099468:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23517 00000100:00100000:1.0:1787826301.099771:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174208:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.099835:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174336:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.099866:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.099893:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.099950:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346174336 00000100:00100000:2.0:1787826301.099955:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346174336 00000100:00100000:2.0:1787826301.100027:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23518 00000100:00100000:2.0:1787826301.100037:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174336:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826301.100612:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174336:12345-0@lo:1000:mdt_io00_003.0 Request processed in 573us (742us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826301.100620:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23518 00000100:00100000:1.0:1787826301.100678:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174336:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.100768:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174464:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.100796:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.105927:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826301.106632:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6511:1874663346174080:192.168.203.56@tcp:105: 00010000:00010000:0.0:1787826301.106659:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826301.106715:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332567936:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 10861us (11155us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.106739:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21906 00000100:00100000:1.0:1787826301.106899:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826301.107234:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346174464 00000100:00100000:3.0:1787826301.107253:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346174464 00000100:00100000:3.0:1787826301.107350:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23519 00000100:00100000:3.0:1787826301.107359:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174464:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826301.107449:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174464:12345-0@lo:1000:mdt_io00_003.0 Request processed in 91us (6652us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.107457:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23519 00010000:00010000:1.0:1787826301.107630:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826301.107657:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174464:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.107772:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174592:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.107808:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.107834:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826301.107885:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346174592 00000100:00100000:3.0:1787826301.107890:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346174592 00000100:00100000:3.0:1787826301.107912:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23520 00000100:00100000:3.0:1787826301.107919:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174592:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826301.108262:0:20179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332568064 00000100:00100000:2.0:1787826301.108268:0:20179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332568064 00002000:00100000:2.0:1787826301.108307:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x51e:0x0] (0->4095) req@ffff9f7849dd1500 x1874663332568064/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:657/0 lens 488/0 e 0 to 0 dl 1787826312 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826301.108317:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd1500 with x1874663332568064 ext(0->4095) 00002000:00010000:2.0:1787826301.108323:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7849dd1500 00000100:00100000:2.0:1787826301.108347:0:20179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27641 00000100:00100000:2.0:1787826301.108355:0:20179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332568064:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:2.0:1787826301.108485:0:20179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332568064:12345-192.168.203.56@tcp:3:cat.0 Request processed in 130us (2568us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787826301.108492:0:20179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27641 00002000:00100000:2.0:1787826301.108498:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x51e:0x0] (0->4095) req@ffff9f7849dd1500 x1874663332568064/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:657/0 lens 488/4536 e 0 to 0 dl 1787826312 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826301.108507:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7849dd1500 with x1874663332568064 ext(0->4095) 00002000:00010000:2.0:1787826301.108510:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7849dd1500 00000100:00100000:3.0:1787826301.110588:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174592:12345-0@lo:1000:mdt_io00_003.0 Request processed in 2659us (2770us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.110602:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23520 00000100:00100000:1.0:1787826301.113371:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174592:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826301.113425:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e48:0x0] 00010000:00010000:1.0:1787826301.113589:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.113611:0:125380:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.113619:0:125380:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.113651:0:125380:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.113665:0:125380:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 9 pid: 35098 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826301.113678:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346174720:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826301.113885:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.118652:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826301.118960:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826301.119180:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332568448 00000100:00100000:0.0:1787826301.119190:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332568448 00010000:00010000:0.0:1787826301.119231:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 10 pid: 35098 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826301.119403:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21907 00000100:00100000:0.0:1787826301.119415:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332568448:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826301.119430:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826301.119437:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 10 pid: 35098 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.119451:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 10 pid: 35098 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.119476:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db8:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:0.0:1787826301.119481:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.119491:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.119500:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.119521:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826301.119544:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a338 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801273 expref: 10 pid: 35098 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826301.119563:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000_UUID:125380:1874663346174720:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826301.119573:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332568448:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 159us (612us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.119586:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21907 00010000:00010000:1.0:1787826301.119588:0:125380:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.119599:0:125380:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.119606:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.119661:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.119670:0:125380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.119680:0:125380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826301.119698:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174848:0@lo:101:mdt_io00_003.0 00000100:00100000:1.0:1787826301.119760:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.119802:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826301.119839:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332568320 00000100:00100000:3.0:1787826301.119847:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332568320 00000100:00100000:3.0:1787826301.119902:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9326 00000100:00100000:3.0:1787826301.119913:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224455:x1874663332568320:12345-192.168.203.56@tcp:35:cat.0 00000100:00100000:0.0:1787826301.120226:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346174848 00000100:00100000:0.0:1787826301.120232:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346174848 00000100:00100000:0.0:1787826301.120273:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23521 00000100:00100000:0.0:1787826301.120280:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:125380:x1874663346174848:12345-0@lo:101:mdt_io00_003.0 00010000:00010000:0.0:1787826301.120289:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826301.120305:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a385 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a37e expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.120336:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a385 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a37e expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.120349:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a385 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a37e expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.120356:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:0.0:1787826301.120359:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787826301.120362:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826301.120421:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346174848:12345-0@lo:101:mdt_io00_003.0 Request processed in 141us (661us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.120427:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23521 00000100:00100000:1.0:1787826301.120582:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174848:0@lo:101:mdt_io00_003.0 00010000:00010000:1.0:1787826301.120597:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a385 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.120610:0:125380:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a385 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826301.120626:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e49:0x0] 00000100:00100000:1.0:1787826301.120671:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174976:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.120699:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.120736:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.120917:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346174976 00000100:00100000:0.0:1787826301.120922:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346174976 00000100:00100000:0.0:1787826301.120966:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23522 00000100:00100000:0.0:1787826301.120974:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346174976:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.121050:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346174976:12345-0@lo:1000:mdt_io00_003.0 Request processed in 76us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.121058:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23522 00000100:00100000:1.0:1787826301.121285:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346174976:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826301.121315:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x192:0x0] 00000100:00100000:1.0:1787826301.121348:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175104:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.121378:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.121403:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.121545:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346175104 00000100:00100000:0.0:1787826301.121550:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346175104 00000100:00100000:0.0:1787826301.121588:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23523 00000100:00100000:0.0:1787826301.121596:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175104:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.121659:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175104:12345-0@lo:1000:mdt_io00_003.0 Request processed in 64us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.121666:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23523 00000100:00100000:1.0:1787826301.121714:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175104:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826301.121744:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e48:0x0] 00010000:00010000:1.0:1787826301.121793:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.121805:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.121814:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826301.121822:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224455:x1874663332568320:12345-192.168.203.56@tcp:35:cat.0 Request processed in 1908us (3175us total) trans 8589957361 rc 0/0 00000100:00100000:3.0:1787826301.121832:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9326 00000100:00100000:1.0:1787826301.121858:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175232:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.121884:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.121908:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826301.121933:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346175232 00000100:00100000:3.0:1787826301.122057:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346175232 00000100:00100000:3.0:1787826301.122227:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23524 00000100:00100000:3.0:1787826301.122236:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175232:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826301.122311:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175232:12345-0@lo:1000:mdt_io00_003.0 Request processed in 77us (426us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.122318:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23524 00000100:00100000:1.0:1787826301.122361:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175232:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826301.122382:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e48:0x0] 00000100:00100000:1.0:1787826301.122403:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175360:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.122438:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.126703:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346175360 00000100:00100000:0.0:1787826301.126730:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346175360 00000100:00100000:1.0:1787826301.126768:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.126789:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23525 00000100:00100000:0.0:1787826301.126797:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf89050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175360:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.126907:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf89050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175360:12345-0@lo:1000:mdt_io00_003.0 Request processed in 111us (4470us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.126912:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23525 00000100:00100000:1.0:1787826301.126967:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175360:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.127050:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175488:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.129616:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.129655:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346175488 00000100:00100000:0.0:1787826301.129665:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346175488 00000100:00100000:1.0:1787826301.129666:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.129739:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23526 00000100:00100000:0.0:1787826301.129750:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175488:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.129881:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175488:12345-0@lo:1000:mdt_io00_003.0 Request processed in 132us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.129889:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23526 00000100:00100000:1.0:1787826301.130071:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175488:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826301.130203:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x192:0x0] 00000100:00100000:1.0:1787826301.130327:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175616:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.130364:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.130521:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.130572:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346175616 00000100:00100000:0.0:1787826301.130577:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346175616 00000100:00100000:2.0:1787826301.130705:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23527 00000100:00100000:2.0:1787826301.130736:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175616:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:2.0:1787826301.130807:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175616:12345-0@lo:1000:mdt_io00_003.0 Request processed in 95us (443us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.130812:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23527 00000100:00100000:1.0:1787826301.130841:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175616:0@lo:1000:mdt_io00_003.0 00000004:00080000:1.0:1787826301.130861:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e48:0x0] 00000100:00100000:1.0:1787826301.130894:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175744:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.130918:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.131086:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346175744 00000100:00100000:0.0:1787826301.131090:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346175744 00000100:00100000:1.0:1787826301.131237:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.131329:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23528 00000100:00100000:0.0:1787826301.131338:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175744:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.131646:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175744:12345-0@lo:1000:mdt_io00_003.0 Request processed in 309us (726us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.131655:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23528 00000100:00100000:1.0:1787826301.131734:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175744:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.131774:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175872:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.131801:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.131856:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.131888:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346175872 00000100:00100000:0.0:1787826301.131893:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346175872 00000100:00100000:0.0:1787826301.131922:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23529 00000100:00100000:0.0:1787826301.131928:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175872:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:0.0:1787826301.132872:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346175872:12345-0@lo:1000:mdt_io00_003.0 Request processed in 939us (1065us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826301.132883:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23529 00000100:00100000:1.0:1787826301.133068:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346175872:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.133322:0:125380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346176000:0@lo:1000:mdt_io00_003.0 00000100:00100000:1.0:1787826301.133356:0:125380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826301.133808:0:125380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826301.133858:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346176000 00000100:00100000:3.0:1787826301.133865:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346176000 00000100:00100000:3.0:1787826301.133958:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23530 00000100:00100000:3.0:1787826301.133967:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346176000:12345-0@lo:1000:mdt_io00_003.0 00000100:00100000:3.0:1787826301.134065:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:125380:x1874663346176000:12345-0@lo:1000:mdt_io00_003.0 Request processed in 99us (708us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826301.134073:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23530 00000100:00100000:1.0:1787826301.134138:0:125380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_003:lustre-MDT0000-mdtlov_UUID:125380:1874663346176000:0@lo:1000:mdt_io00_003.0 00000040:00080000:1.0:1787826301.134583:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].212, 64 off21760 00000004:00080000:1.0:1787826301.134594:0:125380:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8113346487688ns, next at 8114346484614ns, rc = 0 00000040:00080000:1.0:1787826301.134683:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].933, 904 off877864 00000040:00080000:1.0:1787826301.134752:0:125380:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].936, 984 off976848 00000004:00080000:1.0:1787826301.134760:0:125380:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7844e81000 with version 1820 00000004:00080000:0.0:1787826301.134782:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1820 rpc_version 1820 00000020:00100000:0.0:1787826301.134811:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826301.134819:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7849dd0a80 x1874663346176256/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2584/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826301.134838:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346176256:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826301.134883:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.134911:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826301.135307:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346176256 00000100:00100000:1.0:1787826301.135315:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346176256 00000100:00100000:1.0:1787826301.135364:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23531 00000100:00100000:1.0:1787826301.135455:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346176256:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826301.135754:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346176256:12345-0@lo:1000:osp_up1-0.0 Request processed in 382us (869us total) trans 4294985370 rc 0/0 00000100:00100000:1.0:1787826301.135763:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23531 00000100:00100000:0.0:1787826301.136175:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346176256:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787826301.136477:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136489:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136498:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136504:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826301.136509:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136516:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136531:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db8:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:1.0:1787826301.136535:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826301.136538:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136545:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a377 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826301.136568:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e49:0x0] 00010000:00010000:1.0:1787826301.136575:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a385 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136584:0:125380:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a385 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136595:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136602:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136608:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136613:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826301.136617:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136623:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136632:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2200) 00010000:00010000:1.0:1787826301.136635:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826301.136638:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136644:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a362 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826301.136659:0:125380:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fde4195e reply state 000000007bfcf7a9 transno 8589957362 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136677:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136685:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f7961d81200) 00010000:00010000:1.0:1787826301.136687:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826301.136690:0:125380:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:1.0:1787826301.136694:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x192:0x0] 00010000:00010000:1.0:1787826301.136699:0:125380:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136706:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136714:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136793:0:125380:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826301.136799:0:125380:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x192:0x0] 00010000:00010000:1.0:1787826301.136805:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a35b expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136812:0:125380:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a35b expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136821:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136827:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136834:0:125380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136840:0:125380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826301.136843:0:125380:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136849:0:125380:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136857:0:125380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7845cb1d00) 00010000:00010000:1.0:1787826301.136860:0:125380:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826301.136862:0:125380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136868:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a38c lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136897:0:125380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136907:0:125380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136916:0:125380:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136923:0:125380:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136930:0:125380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.136947:0:125380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137234:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826301.137348:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137363:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826301.137368:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826301.137642:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137677:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137684:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826301.137757:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:2.0:1787826301.137777:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137785:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a331 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a346 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137803:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826301.137806:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 4/0,0 mode: PW/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137812:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826301.137815:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 4/0,0 mode: PW/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137822:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 5/0,0 mode: PW/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137832:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db8:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:2.0:1787826301.137835:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826301.137838:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 3/0,0 mode: --/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.137844:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826301.137845:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826301.137847:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826301.137959:0:125380:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787826301.137965:0:125380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a393 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826301.138308:0:125380:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224457:x1874663332567296:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 87106us (87444us total) trans 8589957362 rc 0/0 00000100:00100000:1.0:1787826301.138317:0:125380:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1227 00000100:00100000:3.0:1787826301.140236:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346176384:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826301.140321:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826301.141897:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826301.142060:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346176384 00000100:00100000:1.0:1787826301.142071:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346176384 00000100:00100000:1.0:1787826301.143353:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11054 00000100:00100000:1.0:1787826301.143373:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd82c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3652:x1874663346176384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826301.143392:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826301.143420:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a346 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a331 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.143439:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f794b57ac00) 00010000:00010000:1.0:1787826301.143452:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826301.143589:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd82c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346176384:12345-0@lo:103:ldlm_bl.0 Request processed in 223us (3268us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826301.143597:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11054 00000100:00100000:3.0:1787826301.144661:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346176384:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826301.489763:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a35b expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826301.489786:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a35b expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.489833:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a35b expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.489841:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826301.489864:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68000/0xe21309aa81f4a354 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a35b expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.489885:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a385 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826301.489893:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a385 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.489909:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a385 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.489914:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826301.489918:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf400/0xe21309aa81f4a37e lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a385 expref: -99 pid: 125380 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826301.489929:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957362 is committed 00000004:00080000:0.0:1787826301.489936:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 211 committed 00000001:00080000:0.0:1787826301.489950:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957361 is committed 00000020:00080000:0.0:1787826301.489958:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979079 batchid updated 00000001:00080000:0.0:1787826301.489965:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957359 is committed 00010000:00010000:2.0:1787826301.490138:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.490147:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.490153:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.490160:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826301.490165:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.490171:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.490183:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f7961d81200) 00010000:00010000:2.0:1787826301.490186:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826301.490190:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.490196:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a20b lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787826301.490221:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979079 00000004:00080000:2.0:1787826301.490292:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784c4f4300 with version 1821 00000020:00080000:2.0:1787826301.490304:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979079 cancel update log [0x240000404:0x2:0x0].935: rc = 0 00000020:00080000:2.0:1787826301.490397:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979079 cancel update log [0x200000bd0:0x2:0x0].932: rc = 0 00000020:00080000:2.0:1787826301.490407:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979079 00000100:00100000:2.0:1787826301.490425:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346176640:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826301.490492:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826301.490529:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826301.490534:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346176512:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826301.490553:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826301.490582:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346176640 00000100:00100000:2.0:1787826301.490589:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346176640 00000100:00100000:2.0:1787826301.490632:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346176512 00000100:00100000:2.0:1787826301.490636:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346176512 00000100:00100000:2.0:1787826301.490657:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11055 00000100:00100000:2.0:1787826301.490666:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346176640:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826301.490678:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826301.490696:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a385 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a37e expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.490708:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:2.0:1787826301.490731:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826301.490765:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3655:x1874663346176640:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 99us (272us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.490772:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11055 00000100:00100000:2.0:1787826301.490792:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11056 00000100:00100000:2.0:1787826301.490798:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346176512:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826301.490806:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826301.490817:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a35b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x192:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a354 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826301.490828:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x4ec94fd6 (ffff9f7952ca9200) 00010000:00010000:2.0:1787826301.490834:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826301.490857:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346176512:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 59us (302us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826301.490862:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11056 00000100:00100000:2.0:1787826301.490901:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346176512:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826301.490909:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346176640:0@lo:103:jbd2/dm-0-8.0 00000004:00080000:2.0:1787826301.490932:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1821 rpc_version 1821 00000020:00100000:2.0:1787826301.490951:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826301.490958:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c92680 x1874663346176896/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826301.490974:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346176896:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826301.491006:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826301.491567:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.491579:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.491590:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.491596:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826301.491600:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.491608:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.491619:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f7961d81200) 00010000:00010000:1.0:1787826301.491622:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826301.491626:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.491632:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4a370 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 125380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826301.492774:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346177024:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787826301.492835:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826301.492925:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346177024 00000100:00100000:0.0:1787826301.492933:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346177024 00000100:00100000:0.0:1787826301.492987:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30810 00000100:00100000:0.0:1787826301.492996:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346177024:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787826301.493155:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826301.493173:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346176896 00000100:00100000:2.0:1787826301.493177:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346176896 00000100:00100000:2.0:1787826301.493210:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23532 00000100:00100000:2.0:1787826301.493217:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346176896:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826301.493401:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346176896:12345-0@lo:1000:osp_up1-0.0 Request processed in 183us (2393us total) trans 4294985371 rc 0/0 00000100:00100000:2.0:1787826301.493418:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23532 00000100:00100000:2.0:1787826301.493467:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346176896:0@lo:1000:osp_up1-0.0 00000100:00080000:2.0:1787826301.493501:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00080000:0.0:1787826301.493568:0:35352:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1310 count 1 00010000:00010000:0.0:1787826301.493592:0:35352:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.493612:0:35352:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.493620:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826301.493624:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826301.493629:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 4/0,1 mode: --/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.493655:0:35352:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 5 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.493669:0:35352:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 5 pid: 8759 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826301.493681:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346177152:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826301.493759:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826301.497699:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346177152:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826301.497749:0:35352:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826301.505355:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826301.506146:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332568576 00000100:00100000:1.0:1787826301.506158:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332568576 00010000:00010000:1.0:1787826301.506205:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 6 pid: 8759 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826301.506256:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21908 00000100:00100000:1.0:1787826301.506267:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332568576:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826301.506280:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826301.506287:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 6 pid: 8759 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.506302:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 6 pid: 8759 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.506339:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ddacd1e/0xe21309aa81f4a369 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608012a4 expref: 6 pid: 8759 timeout: 8213 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.506357:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826301.506362:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826301.506368:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.506377:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.506409:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826301.506474:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332568576:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 209us (1127us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826301.506482:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21908 00010000:00010000:0.0:1787826301.506616:0:35352:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.506631:0:35352:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.506641:0:35352:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.506649:0:35352:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826301.506677:0:35352:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826301.506862:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346177024:12345-0@lo:6:osp-syn-0-0.0 Request processed in 13865us (14024us total) trans 4294973844 rc 0/0 00000100:00100000:0.0:1787826301.506870:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30810 00010000:00010000:1.0:1787826301.506930:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826301.506937:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.506946:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826301.506951:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826301.506953:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784b750700/2, rc 0, transno 4294973844 00000100:00100000:3.0:1787826301.506965:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346177024:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1787826301.506966:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.506983:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.506990:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4a39a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826301.507324:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826301.507327:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826301.507329:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826302.125185:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826302.483937:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.484027:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332568704 00000100:00100000:0.0:1787826302.484037:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332568704 00000100:00100000:0.0:1787826302.484100:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68349 00000100:00100000:0.0:1787826302.484140:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224493:x1874663332568704:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826302.484160:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.484243:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3a1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012ab expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.484312:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4a3a8 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.484327:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4a3a8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.484343:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4a3a8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826302.484455:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e48:0x0]: returning size 4096 00000004:00200000:0.0:1787826302.484460:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787826302.484481:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4a3a8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.484509:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ab expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.484531:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4a3a8 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ab expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.484540:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:0.0:1787826302.484543:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826302.484547:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826302.484613:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224493:x1874663332568704:12345-192.168.203.56@tcp:101:rm.0 Request processed in 477us (686us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.484621:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68349 00000100:00100000:3.0:1787826302.491848:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826302.491926:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332568832 00000100:00100000:1.0:1787826302.491936:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332568832 00000100:00100000:1.0:1787826302.492004:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68350 00000100:00100000:1.0:1787826302.492015:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224493:x1874663332568832:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826302.492034:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826302.492057:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a3af lrc: 2/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012b2 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.492135:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.492148:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.492159:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826302.492203:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x192:0x0]: returning size 4096 00000004:00200000:1.0:1787826302.492208:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826302.492253:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a3af lrc: 0/0,0 mode: --/CR res: [0x240000400:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.492268:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.492277:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:1.0:1787826302.492281:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826302.492284:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826302.492343:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224493:x1874663332568832:12345-192.168.203.56@tcp:101:rm.0 Request processed in 330us (503us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826302.492350:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68350 00000100:00100000:2.0:1787826302.506995:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826302.511445:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332568960 00000100:00100000:3.0:1787826302.511462:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332568960 00000100:00100000:3.0:1787826302.511526:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68351 00000100:00100000:3.0:1787826302.511537:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224493:x1874663332568960:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826302.511562:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826302.511586:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a3bd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012b9 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.511645:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.511660:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.511671:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826302.511727:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x190:0x0]: returning size 4096 00000004:00200000:3.0:1787826302.511732:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826302.511770:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4a3bd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.511792:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.511801:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2900) 00010000:00010000:3.0:1787826302.511805:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826302.511808:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826302.511876:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332568960:12345-192.168.203.56@tcp:101:rm.0 Request processed in 341us (4892us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.511884:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68351 00000100:00100000:3.0:1787826302.517339:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.517452:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332569216 00000100:00100000:0.0:1787826302.517463:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332569216 00000100:00100000:0.0:1787826302.517525:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68352 00000100:00100000:0.0:1787826302.517537:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224493:x1874663332569216:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826302.517558:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.517583:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3cb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012c0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826302.517762:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e48:0x0]: returning size 4096 00010000:00010000:0.0:1787826302.517891:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3cb lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012c0 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.517908:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826302.517912:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826302.517918:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3cb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012c0 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.517938:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826302.518296:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332569216:12345-192.168.203.56@tcp:101:rm.0 Request processed in 760us (966us total) trans 4294985372 rc 301/301 00000100:00100000:0.0:1787826302.518304:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68352 00000100:00100000:3.0:1787826302.525954:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826302.526090:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332569344 00000100:00100000:2.0:1787826302.526100:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332569344 00000100:00100000:2.0:1787826302.526184:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9327 00000100:00100000:2.0:1787826302.526196:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332569344:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826302.531496:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332569344:12345-192.168.203.56@tcp:37:rm.0 Request processed in 5292us (5540us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.531510:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9327 00000100:00100000:1.0:1787826302.543768:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826302.544145:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332569472 00000100:00100000:3.0:1787826302.544155:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332569472 00000100:00100000:3.0:1787826302.544213:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9328 00000100:00100000:3.0:1787826302.544241:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd2050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332569472:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826302.549901:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd2050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332569472:12345-192.168.203.56@tcp:37:rm.0 Request processed in 5668us (6132us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826302.549913:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9328 00000100:00100000:3.0:1787826302.554058:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826302.554292:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332569728 00000100:00100000:1.0:1787826302.554301:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332569728 00000100:00100000:1.0:1787826302.554353:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9329 00000100:00100000:1.0:1787826302.554363:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332569728:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:1.0:1787826302.554634:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1191, 344 off1083856 00000040:00080000:1.0:1787826302.554683:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1509, 424 off1524480 00000004:00080000:1.0:1787826302.554688:0:6531:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81c00 with version 6812 00000004:00080000:1.0:1787826302.554719:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6812 rpc_version 6812 00000020:00100000:1.0:1787826302.554745:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826302.554754:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7843092a00 x1874663346177408/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826302.554779:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843092a00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346177408:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826302.554837:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.554873:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.555616:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346177408 00000100:00100000:0.0:1787826302.555625:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346177408 00000100:00100000:0.0:1787826302.555680:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23533 00000100:00100000:0.0:1787826302.555688:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346177408:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826302.555892:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346177408:12345-0@lo:1000:osp_up0-1.0 Request processed in 203us (1053us total) trans 8589957363 rc 0/0 00000100:00100000:0.0:1787826302.555899:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23533 00000100:00100000:1.0:1787826302.556155:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957359 gen 2 00000004:00080000:1.0:1787826302.556161:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957356 committed transno 8589957359 00000004:00080000:1.0:1787826302.556178:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957357 committed transno 8589957359 00000004:00080000:1.0:1787826302.556189:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957358 committed transno 8589957359 00000004:00080000:1.0:1787826302.556198:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957359 committed transno 8589957359 00000100:00100000:1.0:1787826302.556209:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f7843092a00 x1874663346177408/t8589957363(8589957363) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826318 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826302.556245:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843092a00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346177408:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826302.556342:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224493:x1874663332569728:12345-192.168.203.56@tcp:35:rm.0 Request processed in 1979us (2292us total) trans 4294985373 rc 0/0 00000100:00100000:1.0:1787826302.556350:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9329 00000100:00100000:3.0:1787826302.563089:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.563241:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332569984 00000100:00100000:0.0:1787826302.563251:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332569984 00000100:00100000:0.0:1787826302.563304:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68353 00000100:00100000:0.0:1787826302.563314:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224493:x1874663332569984:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826302.563338:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.563368:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3d9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012c7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.563537:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a3e0 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.563550:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a3e0 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.563561:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a3e0 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826302.563572:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e48:0x0]: returning size 4096 00010000:00010000:0.0:1787826302.563666:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3d9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012c7 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.563687:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a3e0 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012c7 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.563696:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:0.0:1787826302.563699:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787826302.563703:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826302.563782:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224493:x1874663332569984:12345-192.168.203.56@tcp:101:rm.0 Request processed in 468us (702us total) trans 4294985374 rc 0/0 00000100:00100000:0.0:1787826302.563789:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68353 00000100:00100000:3.0:1787826302.567341:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.567429:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332570112 00000100:00100000:0.0:1787826302.567439:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332570112 00000100:00100000:0.0:1787826302.567493:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68354 00000100:00100000:0.0:1787826302.567505:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332570112:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:0.0:1787826302.567557:0:35094:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808688128 avail 7345614848 00000100:00100000:0.0:1787826302.567634:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332570112:12345-192.168.203.56@tcp:41:rm.0 Request processed in 132us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.567641:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68354 00000100:00100000:3.0:1787826302.577615:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.578972:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332570240 00000100:00100000:0.0:1787826302.578985:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332570240 00000100:00100000:0.0:1787826302.579057:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68355 00000100:00100000:0.0:1787826302.579069:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224493:x1874663332570240:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826302.579089:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.579145:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3ee lrc: 2/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012ce expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579215:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579229:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579239:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579263:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579271:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579277:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579340:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a403 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579347:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a403 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579353:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a403 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826302.579373:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a403 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579387:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579394:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579410:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579417:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579423:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826302.579427:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579434:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579444:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2900) 00010000:00010000:0.0:1787826302.579448:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826302.579452:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579457:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a3f5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579467:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579472:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579477:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579481:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826302.579483:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579488:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579493:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826302.579495:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826302.579497:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579503:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a3fc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579532:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a3ee lrc: 0/0,0 mode: --/CR res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ce expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579544:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a403 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ce expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.579551:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826302.579553:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826302.579555:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826302.579624:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224493:x1874663332570240:12345-192.168.203.56@tcp:101:rm.0 Request processed in 558us (2019us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.579632:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68355 00000100:00100000:2.0:1787826302.593933:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826302.594019:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332570368 00000100:00100000:3.0:1787826302.594030:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332570368 00000100:00100000:3.0:1787826302.594093:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68356 00000100:00100000:3.0:1787826302.594235:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224493:x1874663332570368:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826302.594380:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826302.594405:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4a40a lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012d5 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.594472:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a411 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.594484:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a411 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.595596:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a411 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826302.595644:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e49:0x0]: returning size 6 00000004:00200000:3.0:1787826302.595648:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:3.0:1787826302.595686:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4a40a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012d5 expref: 14 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.595701:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a411 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012d5 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.595707:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f79460a2a00) 00010000:00010000:3.0:1787826302.595720:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826302.595725:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826302.595856:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224493:x1874663332570368:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1752us (1931us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.595864:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68356 00000100:00100000:0.0:1787826302.605275:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.605321:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826302.605692:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332570496 00000100:00100000:3.0:1787826302.605703:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332570496 00000100:00100000:3.0:1787826302.605794:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11057 00000100:00100000:3.0:1787826302.605803:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332570496:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826302.605820:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826302.605844:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a403 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ce expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.605861:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:3.0:1787826302.605873:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826302.605925:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332570496:12345-192.168.203.56@tcp:103:rm.0 Request processed in 123us (660us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.605933:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11057 00000100:00100000:1.0:1787826302.609727:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332570752 00000100:00100000:1.0:1787826302.609743:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332570752 00000100:00100000:1.0:1787826302.609810:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68357 00000100:00100000:1.0:1787826302.609821:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224493:x1874663332570752:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826302.609875:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826302.609902:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a411 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e49:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012d5 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.609934:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f79460a2a00) 00010000:00010000:1.0:1787826302.609947:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826302.609992:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.610001:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.610010:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.610036:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346177536:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1787826302.610125:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.610158:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.611662:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346177536 00000100:00100000:2.0:1787826302.611675:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346177536 00000100:00100000:2.0:1787826302.611752:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23534 00000100:00100000:2.0:1787826302.611763:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:17813:x1874663346177536:12345-0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826302.611784:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826302.611805:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a418 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.611833:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.611843:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 4/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a418 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.611877:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.611889:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 8 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.611902:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346177664:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826302.611998:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.615363:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346177664:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826302.615399:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 3/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a418 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.615422:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2900) 00010000:00010000:2.0:1787826302.615425:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826302.615429:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 3/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a418 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.615438:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826302.615717:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:17813:x1874663346177536:12345-0@lo:101:mdt00_005.0 Request processed in 3947us (5602us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.615726:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23534 00000100:00100000:3.0:1787826302.615915:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346177536:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787826302.616001:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.616012:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.618388:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.623972:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332570880 00000100:00100000:0.0:1787826302.623988:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332570880 00010000:00010000:1.0:1787826302.624149:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 9 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.624184:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21909 00000100:00100000:1.0:1787826302.624193:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332570880:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826302.624207:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826302.624214:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 9 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.624226:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 9 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.624249:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a3c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x190:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b9 expref: 9 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.624264:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2900) 00010000:00010000:1.0:1787826302.624272:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a418 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.624283:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a418 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.624292:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a418 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.624325:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a418 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.624338:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346177792:0@lo:105: 00000100:00100000:1.0:1787826302.624398:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.624706:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.624754:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346177792 00000100:00100000:2.0:1787826302.624759:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346177792 00000100:00100000:2.0:1787826302.624795:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7841 00000100:00100000:2.0:1787826302.624802:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346177792:12345-0@lo:105: 00010000:00010000:2.0:1787826302.624813:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.624823:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.624831:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.624838:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.624845:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.624854:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.624876:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece800) 00000100:00100000:2.0:1787826302.624912:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346177792:12345-0@lo:105: Request processed in 110us (513us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.624918:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7841 00010000:00010000:1.0:1787826302.624996:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.625006:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826302.625029:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x190:0x0] 00010000:00010000:1.0:1787826302.625060:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.625067:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.625076:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.625091:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346177920:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1787826302.625136:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.625150:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.625175:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346177792:0@lo:105: 00010000:00010000:1.0:1787826302.625186:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826302.625222:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332570880:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1030us (6843us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826302.625229:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21909 00000100:00100000:0.0:1787826302.625371:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346177920 00000100:00100000:0.0:1787826302.625375:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346177920 00000100:00100000:0.0:1787826302.625415:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23535 00000100:00100000:0.0:1787826302.625422:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:17813:x1874663346177920:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826302.625436:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.625450:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a42d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a426 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.625467:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a42d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a426 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.625478:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a42d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a426 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.625667:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826302.625670:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826302.625673:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826302.625721:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346177920:12345-0@lo:101:mdt00_005.0 Request processed in 299us (583us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.625727:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23535 00000100:00100000:1.0:1787826302.626610:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346177920:0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826302.626748:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a42d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.626761:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a42d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826302.626781:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x190:0x0] 00000100:00100000:1.0:1787826302.626821:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346178048:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826302.626861:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.626965:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.631044:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346178048 00000100:00100000:2.0:1787826302.631060:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346178048 00000100:00100000:2.0:1787826302.631187:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23536 00000100:00100000:2.0:1787826302.631199:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346178048:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787826302.631339:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346178048:12345-0@lo:1000:mdt00_005.0 Request processed in 142us (4477us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.631348:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23536 00000100:00100000:1.0:1787826302.631450:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346178048:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826302.631543:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346178176:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826302.631581:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.632668:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.632897:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346178176 00000100:00100000:0.0:1787826302.632905:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346178176 00000100:00100000:0.0:1787826302.632958:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23537 00000100:00100000:0.0:1787826302.632967:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346178176:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826302.633067:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346178176:12345-0@lo:1000:mdt00_005.0 Request processed in 101us (1485us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.633074:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23537 00000100:00100000:0.0:1787826302.633128:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346178176:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826302.633185:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346178304:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826302.633215:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.633227:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.633239:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346178304 00000100:00100000:0.0:1787826302.633242:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346178304 00000100:00100000:0.0:1787826302.633260:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23538 00000100:00100000:0.0:1787826302.633266:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346178304:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826302.633329:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346178304:12345-0@lo:1000:mdt00_005.0 Request processed in 63us (113us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826302.633335:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23538 00000100:00100000:0.0:1787826302.633372:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346178304:0@lo:1000:mdt00_005.0 00010000:00010000:0.0:1787826302.633435:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633446:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633455:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.633465:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851885500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346178432:0@lo:101:mdt00_005.0 00000100:00100000:0.0:1787826302.633494:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.633506:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.633516:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346178432 00000100:00100000:0.0:1787826302.633519:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346178432 00000100:00100000:0.0:1787826302.633535:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23539 00000100:00100000:0.0:1787826302.633541:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346178432:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826302.633549:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.633562:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a43b lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a434 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633583:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a43b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a434 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633595:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a43b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a434 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633603:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:0.0:1787826302.633606:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826302.633610:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826302.633849:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:17813:x1874663346178432:12345-0@lo:101:mdt00_005.0 Request processed in 306us (353us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.633856:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23539 00000100:00100000:0.0:1787826302.633885:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851885500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346178432:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826302.633897:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633907:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633939:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633947:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.633955:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787826302.634258:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].221, 64 off22336 00000004:00080000:0.0:1787826302.634272:0:17813:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8114846165296ns, next at 8115846162641ns, rc = 0 00000040:00080000:0.0:1787826302.634302:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1192, 384 off1084240 00000040:00080000:0.0:1787826302.634349:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1510, 464 off1524944 00000004:00080000:0.0:1787826302.634355:0:17813:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852405cc0 with version 6813 00000004:00080000:0.0:1787826302.634373:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6813 rpc_version 6813 00000020:00100000:0.0:1787826302.634392:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826302.634399:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7851885500 x1874663346178688/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826302.634588:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851885500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346178688:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826302.634639:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.634657:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852405240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.634676:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346178688 00000100:00100000:0.0:1787826302.634680:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346178688 00000100:00100000:0.0:1787826302.634720:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23540 00000100:00100000:0.0:1787826302.634727:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346178688:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826302.634963:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346178688:12345-0@lo:1000:osp_up0-1.0 Request processed in 235us (322us total) trans 8589957364 rc 0/0 00000100:00100000:0.0:1787826302.634970:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23540 00000100:00100000:0.0:1787826302.635019:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851885500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346178688:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826302.635051:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635063:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635072:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635080:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635086:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635092:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635130:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826302.635148:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635154:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635161:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635166:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826302.635170:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635177:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635188:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:0.0:1787826302.635191:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826302.635194:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635200:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4a442 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635214:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635221:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635231:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635246:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787826302.635257:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x190:0x0] 00010000:00010000:0.0:1787826302.635264:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635271:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635277:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635289:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787826302.635292:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x190:0x0] 00010000:00010000:0.0:1787826302.635297:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a42d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635303:0:17813:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a42d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.635363:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224493:x1874663332570752:12345-192.168.203.56@tcp:36:rm.0 Request processed in 25543us (30040us total) trans 4294985375 rc 0/0 00000100:00100000:0.0:1787826302.635371:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68357 00010000:00010000:0.0:1787826302.635393:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787826302.635399:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635864:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826302.635868:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635876:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635886:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:0.0:1787826302.635888:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826302.635891:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635896:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a449 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e49:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635913:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826302.635917:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.635976:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826302.635981:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826302.635989:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636017:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636028:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826302.636094:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346178816:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826302.636160:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826302.636234:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636242:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43e00/0xe21309aa81f4a434 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a43b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636257:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826302.636261:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636267:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826302.636271:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826302.636277:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636296:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636302:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826302.636318:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636326:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4a418 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a41f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636343:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826302.636345:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826302.636347:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826302.636353:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826302.636359:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826302.636607:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851885f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346178944:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826302.636650:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.636687:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346178816 00000100:00100000:0.0:1787826302.636693:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346178816 00000100:00100000:0.0:1787826302.636751:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346178944 00000100:00100000:0.0:1787826302.636756:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346178944 00000100:00100000:0.0:1787826302.636776:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11058 00000100:00100000:0.0:1787826302.636784:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346178816:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826302.636795:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826302.636813:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a43b lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e49:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a434 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.636826:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e49:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:0.0:1787826302.636833:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826302.637024:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11059 00000100:00100000:3.0:1787826302.637036:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd83c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346178944:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826302.637044:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826302.637056:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a41f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x190:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a418 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.637067:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f79460a2900) 00010000:00010000:3.0:1787826302.637074:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826302.637130:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd83c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346178944:12345-0@lo:103:ldlm_bl.0 Request processed in 98us (479us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.637139:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11059 00000100:00100000:3.0:1787826302.642443:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826302.642999:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826302.644372:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332571264 00000100:00100000:1.0:1787826302.644387:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332571264 00000100:00100000:1.0:1787826302.647130:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9330 00000100:00100000:1.0:1787826302.647148:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224493:x1874663332571264:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:1.0:1787826302.647377:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:224493:x1874663332571264:12345-192.168.203.56@tcp:35:rm.0 Request processed in 234us (4378us total) trans 4294985376 rc 0/0 00000100:00100000:1.0:1787826302.647385:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9330 00000100:00100000:0.0:1787826302.647539:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346178816:12345-0@lo:103:ldlm_bl.0 Request processed in 10752us (11376us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.647549:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11058 00000100:00100000:0.0:1787826302.647602:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332571008 00000100:00100000:0.0:1787826302.647609:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332571008 00000100:00100000:0.0:1787826302.647646:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11060 00000100:00100000:0.0:1787826302.647653:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332571008:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826302.647666:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826302.647691:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a27b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801257 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.647719:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2800) 00010000:00010000:0.0:1787826302.647724:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826302.647747:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332571008:12345-192.168.203.56@tcp:103:rm.0 Request processed in 95us (5316us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.647754:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11060 00000100:00100000:0.0:1787826302.647822:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851885f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346178944:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826302.654665:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346178816:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826302.656167:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826302.656208:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332571520 00000100:00100000:1.0:1787826302.656217:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332571520 00000100:00100000:1.0:1787826302.656271:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68358 00000100:00100000:1.0:1787826302.656282:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:224493:x1874663332571520:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826302.656339:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826302.656366:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a3e0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e48:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012c7 expref: 14 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656386:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4a3a8 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ab expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656399:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4a289 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080125e expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656424:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:1.0:1787826302.656429:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826302.656472:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656480:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656606:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.656638:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179072:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787826302.656683:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.656723:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.656746:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346179072 00000100:00100000:1.0:1787826302.656750:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346179072 00000100:00100000:1.0:1787826302.656788:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23541 00000100:00100000:1.0:1787826302.656795:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346179072:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787826302.656808:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826302.656821:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a450 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656841:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 6 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656850:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656858:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a450 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656877:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.656889:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 7 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.656900:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346179200:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826302.657029:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 6 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.657039:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 6 pid: 11629 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.657048:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb3100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346179328:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826302.657066:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.663079:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.663122:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.663149:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332571648 00000100:00100000:0.0:1787826302.663157:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332571648 00010000:00010000:0.0:1787826302.663200:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 8 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.663232:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332571776 00000100:00100000:0.0:1787826302.663235:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332571776 00010000:00010000:0.0:1787826302.663248:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 7 pid: 11629 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.663269:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21910 00000100:00100000:0.0:1787826302.663279:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332571648:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826302.663294:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826302.663301:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 8 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.663314:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 8 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.663338:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826302.663345:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a450 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.663356:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826302.663731:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332571648:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 450us (658us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.663741:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21910 00000100:00100000:0.0:1787826302.663762:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21911 00000100:00100000:0.0:1787826302.663771:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332571776:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826302.663780:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826302.663785:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 7 pid: 11629 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.663796:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 7 pid: 11629 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.663810:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826302.663815:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a450 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.663825:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a450 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.663834:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a450 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.663866:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a450 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.663880:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851884e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346179456:0@lo:105: 00000100:00100000:0.0:1787826302.663934:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.663972:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78524056c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.664059:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346179456 00000100:00100000:0.0:1787826302.664063:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346179456 00000100:00100000:0.0:1787826302.664092:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7842 00000100:00100000:0.0:1787826302.664099:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346179456:12345-0@lo:105: 00010000:00010000:0.0:1787826302.664127:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.664137:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.664146:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.664154:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.664161:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.664170:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.664185:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece400) 00000100:00100000:0.0:1787826302.664218:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346179456:12345-0@lo:105: Request processed in 119us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.664224:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7842 00000100:00100000:0.0:1787826302.664274:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851884e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346179456:0@lo:105: 00010000:00010000:0.0:1787826302.664292:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826302.664316:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332571776:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 546us (1193us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.664323:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21911 00010000:00010000:2.0:1787826302.665841:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a2c8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080127a expref: 7 pid: 6528 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.665864:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346179200:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826302.665877:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a25f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801250 expref: 6 pid: 11629 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.665887:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb3100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346179328:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826302.665934:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a450 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.665943:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826302.665946:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826302.665949:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:2.0:1787826302.666140:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6528:x1874663346179072:12345-0@lo:101:mdt00_001.0 Request processed in 9344us (9455us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.666148:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23541 00000100:00100000:0.0:1787826302.666275:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179072:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787826302.666293:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826302.666315:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826302.666361:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.666369:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.666378:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.666397:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851884e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179584:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787826302.666448:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.666476:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.666693:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346179584 00000100:00100000:2.0:1787826302.666700:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346179584 00000100:00100000:2.0:1787826302.666758:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23542 00000100:00100000:2.0:1787826302.666767:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6528:x1874663346179584:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826302.666782:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826302.666799:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a465 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a45e expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.666822:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a465 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a45e expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.666833:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a465 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a45e expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.666841:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9b00) 00010000:00010000:2.0:1787826302.666843:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826302.666846:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826302.666881:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346179584:12345-0@lo:101:mdt00_001.0 Request processed in 116us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.666887:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23542 00000100:00100000:2.0:1787826302.667052:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851884e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179584:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826302.667066:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.667076:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826302.667092:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826302.667143:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179712:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826302.667170:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.667185:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.667200:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346179712 00000100:00100000:2.0:1787826302.667203:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346179712 00000100:00100000:2.0:1787826302.667225:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23543 00000100:00100000:2.0:1787826302.667231:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346179712:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826302.667287:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346179712:12345-0@lo:1000:mdt00_001.0 Request processed in 56us (116us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.667294:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23543 00000100:00100000:2.0:1787826302.667318:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179712:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826302.667356:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179840:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826302.667378:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.667393:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.669244:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346179840 00000100:00100000:2.0:1787826302.669258:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346179840 00000100:00100000:2.0:1787826302.669317:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23544 00000100:00100000:2.0:1787826302.669327:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346179840:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826302.669588:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346179840:12345-0@lo:1000:mdt00_001.0 Request processed in 263us (2208us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826302.669596:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23544 00000100:00100000:2.0:1787826302.669661:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179840:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787826302.669749:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.669759:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.669768:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.669791:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179968:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787826302.669826:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.669847:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.669859:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346179968 00000100:00100000:2.0:1787826302.669863:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346179968 00000100:00100000:2.0:1787826302.669885:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23545 00000100:00100000:2.0:1787826302.669890:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346179968:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826302.669899:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826302.669913:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a46c expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.670170:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.670181:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a46c expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.670203:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.670217:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 6 pid: 6529 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.670228:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346180096:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826302.670284:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.678727:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346180096:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826302.678769:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a46c expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.678782:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2800) 00010000:00010000:0.0:1787826302.678786:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826302.678791:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a46c expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.678802:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:0.0:1787826302.678858:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:6528:x1874663346179968:12345-0@lo:101:mdt00_001.0 Request processed in 8965us (9030us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.678866:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23545 00000100:00100000:0.0:1787826302.678916:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346179968:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787826302.678944:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.678956:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.682650:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826302.682855:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332571904 00000100:00100000:2.0:1787826302.682867:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332571904 00010000:00010000:2.0:1787826302.682913:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 7 pid: 6529 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.682957:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21912 00000100:00100000:2.0:1787826302.682968:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332571904:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826302.682986:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826302.684076:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 7 pid: 6529 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684101:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 7 pid: 6529 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684137:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a2e4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801281 expref: 7 pid: 6529 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684436:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2800) 00010000:00010000:2.0:1787826302.684445:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a46c expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684457:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a46c expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684466:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a46c expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.684595:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a46c expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826302.684614:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b59f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346180224:0@lo:105: 00000100:00100000:3.0:1787826302.684678:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826302.684735:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.684794:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346180224 00000100:00100000:2.0:1787826302.684801:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346180224 00000100:00100000:2.0:1787826302.684855:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7843 00000100:00100000:2.0:1787826302.684867:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346180224:12345-0@lo:105: 00010000:00010000:2.0:1787826302.684880:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684891:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684899:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684907:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684924:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684932:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.684959:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eee00) 00000100:00100000:2.0:1787826302.685006:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346180224:12345-0@lo:105: Request processed in 142us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.685014:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7843 00010000:00010000:1.0:1787826302.685130:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.685141:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.685179:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.685191:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826302.685199:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b59f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346180224:0@lo:105: 00010000:00010000:1.0:1787826302.685200:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.685220:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826302.685224:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.685252:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.685260:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826302.685268:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332571904:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2300us (2625us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.685278:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21912 00010000:00010000:1.0:1787826302.685293:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.685308:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 11 pid: 17813 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.685320:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb1180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6528:1874663346180352:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826302.685361:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.689856:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.689896:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332572032 00000100:00100000:0.0:1787826302.689906:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332572032 00010000:00010000:0.0:1787826302.689959:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 12 pid: 17813 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.689999:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21913 00000100:00100000:0.0:1787826302.690010:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332572032:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826302.690024:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826302.690031:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 12 pid: 17813 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.690044:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 12 pid: 17813 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.690067:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:0.0:1787826302.690073:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.690082:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.690089:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.690125:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826302.690173:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332572032:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 165us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.690181:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21913 00010000:00010000:2.0:1787826302.694671:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a3b6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012b2 expref: 11 pid: 17813 timeout: 8214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.694698:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb1180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6528:1874663346180352:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826302.694740:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.695070:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.695080:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.695156:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.695164:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.695176:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826302.695204:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b59f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346180480:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787826302.695285:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826302.695322:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.695870:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346180480 00000100:00100000:1.0:1787826302.695878:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346180480 00000100:00100000:1.0:1787826302.695935:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23546 00000100:00100000:1.0:1787826302.695945:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:6528:x1874663346180480:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787826302.695961:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826302.695979:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a48f lrc: 2/0,0 mode: --/EX res: [0x200000401:0x190:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a488 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.696179:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a48f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a488 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.696195:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a48f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a488 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.696210:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:1.0:1787826302.696213:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826302.696217:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826302.696760:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:6528:x1874663346180480:12345-0@lo:101:mdt00_001.0 Request processed in 815us (1473us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.696762:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b59f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346180480:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787826302.696768:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23546 00010000:00010000:3.0:1787826302.696779:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.696790:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826302.697835:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346180608:0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787826302.697893:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826302.697925:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.698388:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346180608 00000100:00100000:2.0:1787826302.698396:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346180608 00000100:00100000:2.0:1787826302.698443:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23547 00000100:00100000:2.0:1787826302.698453:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:6528:x1874663346180608:12345-0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787826302.703376:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:6528:x1874663346180608:12345-0@lo:403:mdt00_001.0 Request processed in 4914us (5471us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.703392:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23547 00000100:00100000:2.0:1787826302.703593:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346180608:0@lo:403:mdt00_001.0 00000040:00080000:3.0:1787826302.705629:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1193, 952 off1085192 00000040:00080000:3.0:1787826302.705846:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1511, 1032 off1525976 00000004:00080000:3.0:1787826302.705853:0:6528:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78500aae40 with version 6814 00000004:00080000:0.0:1787826302.706070:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6814 rpc_version 6814 00000020:00100000:0.0:1787826302.706134:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826302.706146:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c92d80 x1874663346180864/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826302.706177:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346180864:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826302.706315:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.706396:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346180864 00000100:00100000:0.0:1787826302.706402:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.706404:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346180864 00000100:00100000:2.0:1787826302.706456:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23548 00000100:00100000:2.0:1787826302.706464:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346180864:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826302.707086:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346180864:12345-0@lo:1000:osp_up0-1.0 Request processed in 620us (769us total) trans 8589957365 rc 0/0 00000100:00100000:2.0:1787826302.707096:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23548 00000100:00100000:0.0:1787826302.707206:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346180864:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787826302.707328:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e48:0x0]: returning size 4096 00010000:00010000:3.0:1787826302.707349:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707358:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707366:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707372:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826302.707377:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707385:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707400:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x192:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:3.0:1787826302.707403:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826302.707408:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707413:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a481 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707433:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707440:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707448:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707515:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826302.707524:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x190:0x0] 00010000:00010000:3.0:1787826302.707533:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707540:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707546:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707551:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826302.707553:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707558:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707565:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826302.707568:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826302.707570:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707575:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a47a lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707584:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707591:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707598:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707684:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826302.707691:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707698:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826302.707702:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826302.707718:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707746:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707753:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826302.707786:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707793:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68a00/0xe21309aa81f4a488 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a48f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707812:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826302.707816:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707822:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826302.707830:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826302.707838:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707842:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787826302.707854:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826302.707854:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826302.707860:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826302.707861:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826302.707868:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707875:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707876:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707883:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4a46c lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a473 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707884:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707895:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826302.707896:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787826302.707898:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826302.707900:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826302.707904:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826302.707904:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826302.707908:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707911:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826302.707916:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.707918:0:6528:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707938:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707942:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826302.707947:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707953:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4a450 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a457 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.707961:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826302.707962:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826302.707964:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826302.707983:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346181120:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826302.708011:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224493:x1874663332571520:12345-192.168.203.56@tcp:36:rm.0 Request processed in 51729us (51850us total) trans 4294985377 rc 0/0 00000100:00100000:3.0:1787826302.708018:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68358 00000100:00100000:1.0:1787826302.708022:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.708047:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826302.708052:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346181248:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826302.708074:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.708129:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346181120 00000100:00100000:1.0:1787826302.708135:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346181120 00000100:00100000:1.0:1787826302.708174:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346181248 00000100:00100000:1.0:1787826302.708177:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346181248 00000100:00100000:1.0:1787826302.708198:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11061 00000100:00100000:1.0:1787826302.708206:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+16:3652:x1874663346181120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826302.708220:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826302.708237:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4a473 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e48:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a46c expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.708260:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2800) 00010000:00010000:1.0:1787826302.708267:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826302.708294:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346181120:12345-0@lo:103:ldlm_bl.0 Request processed in 90us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826302.708300:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11061 00000100:00100000:1.0:1787826302.708314:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11062 00000100:00100000:1.0:1787826302.708320:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd81450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+14:3652:x1874663346181248:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826302.708327:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826302.708338:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a457 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a450 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.708347:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826302.708350:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826302.708370:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd81450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346181248:12345-0@lo:103:ldlm_bl.0 Request processed in 51us (296us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826302.708375:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11062 00000100:00100000:1.0:1787826302.708434:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346181248:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826302.708441:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346181120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826302.708863:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346180992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826302.708896:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.709214:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346180992 00000100:00100000:1.0:1787826302.709219:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346180992 00000100:00100000:1.0:1787826302.709399:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11063 00000100:00100000:1.0:1787826302.709409:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346180992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826302.709418:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826302.709432:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a48f lrc: 0/0,0 mode: --/EX res: [0x200000401:0x190:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a488 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.709446:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:1.0:1787826302.709453:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826302.709582:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346180992:12345-0@lo:103:ldlm_bl.0 Request processed in 174us (683us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826302.709588:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11063 00000100:00100000:2.0:1787826302.709669:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346180992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826302.814037:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826302.814204:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332572160 00000100:00100000:3.0:1787826302.814214:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332572160 00000100:00100000:3.0:1787826302.814288:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68359 00000100:00100000:3.0:1787826302.814298:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224501:x1874663332572160:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:3.0:1787826302.814353:0:35080:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808688128 avail 7345614848 00000100:00100000:3.0:1787826302.814433:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224501:x1874663332572160:12345-192.168.203.56@tcp:41:stat.0 Request processed in 137us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.814442:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68359 00000100:00100000:3.0:1787826302.878543:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.878704:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332572288 00000100:00100000:0.0:1787826302.878726:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332572288 00000100:00100000:0.0:1787826302.878811:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68360 00000100:00100000:0.0:1787826302.878825:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224505:x1874663332572288:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826302.878861:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.878888:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a496 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012dc expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.879320:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.879404:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.879427:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826302.879493:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826302.879499:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826302.879527:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.879560:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a496 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.879584:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.879595:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826302.879598:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826302.879603:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826302.879871:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224505:x1874663332572288:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 1005us (1302us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.879881:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68360 00000100:00100000:1.0:1787826302.885787:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.885872:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332572416 00000100:00100000:0.0:1787826302.885889:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332572416 00000100:00100000:0.0:1787826302.885984:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68361 00000100:00100000:0.0:1787826302.885998:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224505:x1874663332572416:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826302.886026:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.886054:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a4a4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012e3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.886164:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.886182:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.886195:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.886207:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.886219:0:35094:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a465 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a45e expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.886227:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.886273:0:35094:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a465 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a45e expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.886291:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000_UUID:35094:1874663346181376:0@lo:104: 00000100:00100000:0.0:1787826302.886381:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.886441:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.887727:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346181376 00000100:00100000:1.0:1787826302.887743:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346181376 00000100:00100000:1.0:1787826302.887815:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7844 00000100:00100000:1.0:1787826302.887827:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35094:x1874663346181376:12345-0@lo:104: 00010000:00010000:1.0:1787826302.887845:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.887962:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826302.887971:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35094:x1874663346181376:12345-0@lo:104: Request processed in 146us (1588us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826302.887979:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7844 00000100:00100000:3.0:1787826302.888050:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000_UUID:35094:1874663346181376:0@lo:104: 00010000:00010000:3.0:1787826302.888086:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.889615:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826302.889630:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.889642:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826302.889650:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826302.889660:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.889726:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.889734:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826302.889767:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.889774:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4a45e lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a465 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.889800:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826302.889802:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826302.889805:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826302.889823:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346181504:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826302.889918:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.890094:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346181504 00000100:00100000:1.0:1787826302.890136:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346181504 00010000:00010000:1.0:1787826302.890182:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a465 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a45e expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.890227:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21914 00000100:00100000:1.0:1787826302.890237:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346181504:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826302.890255:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787826302.898738:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957365 is committed 00010000:00010000:1.0:1787826302.899605:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4a465 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a45e expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.899636:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9b00) 00010000:00010000:1.0:1787826302.899644:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.899656:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.899664:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.901490:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826302.902173:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346181504:12345-0@lo:103:ldlm_bl.0 Request processed in 11928us (12246us total) trans 0 rc 0/0 00010000:00010000:3.0:1787826302.902178:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.902187:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21914 00010000:00010000:3.0:1787826302.902189:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.902968:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.902982:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.902988:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.902996:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903003:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826302.903008:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903017:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903039:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826302.903042:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826302.903048:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903055:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a4ab lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903074:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903093:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903098:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903136:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826302.903140:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903146:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903155:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9b00) 00010000:00010000:3.0:1787826302.903157:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826302.903160:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903166:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a4b2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903641:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a4a4 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012e3 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.903652:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346181504:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826302.903655:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826302.903657:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826302.903661:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a4a4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012e3 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.903672:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:3.0:1787826302.903785:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224505:x1874663332572416:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 17789us (18006us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826302.903795:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68361 00000100:00100000:1.0:1787826302.908166:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826302.908400:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332572544 00000100:00100000:0.0:1787826302.908412:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332572544 00000100:00100000:0.0:1787826302.908488:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68362 00000100:00100000:0.0:1787826302.908502:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224505:x1874663332572544:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826302.908529:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.908557:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a4b9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012ea expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.908649:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.908665:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.908680:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826302.908760:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826302.908765:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826302.908790:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.908819:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a4b9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.909239:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.909250:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826302.909253:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826302.909259:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826302.909439:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224505:x1874663332572544:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 929us (1278us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.909449:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68362 00000100:00100000:1.0:1787826302.919437:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826302.919630:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332572800 00000100:00100000:2.0:1787826302.919645:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332572800 00000100:00100000:2.0:1787826302.919738:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68363 00000100:00100000:2.0:1787826302.919749:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224505:x1874663332572800:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:2.0:1787826302.919921:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346181632:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826302.919995:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.920038:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.920314:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346181632 00000100:00100000:1.0:1787826302.920319:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346181632 00000100:00100000:1.0:1787826302.920368:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23549 00000100:00100000:1.0:1787826302.920382:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346181632:12345-0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826302.920471:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346181632:12345-0@lo:1000:mdt00_007.0 Request processed in 90us (475us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826302.920478:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23549 00000100:00100000:2.0:1787826302.920811:0:24703:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957365 gen 2 00000004:00080000:2.0:1787826302.920817:0:24703:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957363 committed transno 8589957365 00000004:00080000:2.0:1787826302.920936:0:24703:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957364 committed transno 8589957365 00000004:00080000:2.0:1787826302.920950:0:24703:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957365 committed transno 8589957365 00000100:00100000:2.0:1787826302.920969:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346181632:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826302.921030:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346181760:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826302.921060:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.921087:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.921348:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346181760 00000100:00100000:1.0:1787826302.921354:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346181760 00000100:00100000:1.0:1787826302.921399:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23550 00000100:00100000:1.0:1787826302.921408:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346181760:12345-0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826302.921524:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346181760:12345-0@lo:1000:mdt00_007.0 Request processed in 117us (462us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826302.921531:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23550 00000100:00100000:3.0:1787826302.921651:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346181760:0@lo:1000:mdt00_007.0 00010000:00010000:3.0:1787826302.921733:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.921745:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.921756:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826302.921769:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346181888:0@lo:101:mdt00_007.0 00000100:00100000:3.0:1787826302.921800:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826302.921833:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.922134:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346181888 00000100:00100000:1.0:1787826302.922140:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346181888 00000100:00100000:1.0:1787826302.922176:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23551 00000100:00100000:1.0:1787826302.922183:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346181888:12345-0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826302.922194:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826302.922211:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a4c7 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.922240:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.922250:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.922259:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.922285:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.922299:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 6 pid: 35098 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.922317:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346182016:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826302.922774:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.922785:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 6 pid: 35092 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.922797:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346182144:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826302.922840:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.929906:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826302.929933:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826302.930047:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332572928 00000100:00100000:2.0:1787826302.930058:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332572928 00010000:00010000:2.0:1787826302.930150:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 7 pid: 35098 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.930188:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332573056 00000100:00100000:2.0:1787826302.930191:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332573056 00010000:00010000:2.0:1787826302.930208:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 7 pid: 35092 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.930240:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21915 00000100:00100000:2.0:1787826302.930259:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332572928:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826302.930279:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826302.930287:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 7 pid: 35098 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.930302:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 7 pid: 35098 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.930331:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826302.930350:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.930362:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826302.930434:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332572928:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 177us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.930446:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21915 00000100:00100000:2.0:1787826302.930470:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21916 00000100:00100000:2.0:1787826302.930477:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332573056:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826302.930485:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826302.930490:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 7 pid: 35092 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.930501:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 7 pid: 35092 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.930513:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826302.930518:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.930528:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.930538:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.930576:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826302.930592:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843092a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346182272:0@lo:105: 00000100:00100000:2.0:1787826302.930643:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.930954:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f43c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.930986:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346182272 00000100:00100000:2.0:1787826302.930992:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346182272 00000100:00100000:2.0:1787826302.931027:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7845 00000100:00100000:2.0:1787826302.931034:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346182272:12345-0@lo:105: 00010000:00010000:2.0:1787826302.931047:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.931058:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.931067:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.931075:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.931083:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.931092:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826302.931135:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69000) 00000100:00100000:2.0:1787826302.931176:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346182272:12345-0@lo:105: Request processed in 142us (532us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.931183:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7845 00000100:00100000:2.0:1787826302.931589:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843092a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346182272:0@lo:105: 00010000:00010000:2.0:1787826302.931610:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826302.931645:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332573056:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1166us (1708us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.931653:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21916 00010000:00010000:0.0:1787826302.931839:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a4c0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012ea expref: 6 pid: 35098 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.931866:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346182016:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826302.931882:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4a49d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608012dc expref: 6 pid: 35092 timeout: 8215 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826302.931891:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346182144:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826302.931921:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.931931:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826302.931934:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826302.931937:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:0.0:1787826302.931999:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:24703:x1874663346181888:12345-0@lo:101:mdt00_007.0 Request processed in 9816us (10197us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.932008:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23551 00000100:00100000:3.0:1787826302.933209:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346181888:0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826302.933632:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826302.933670:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826302.933822:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.936504:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.936529:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826302.936570:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182400:0@lo:101:mdt00_007.0 00000100:00100000:3.0:1787826302.936666:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826302.936931:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.943549:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346182400 00000100:00100000:0.0:1787826302.943572:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346182400 00000100:00100000:0.0:1787826302.943671:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23552 00000100:00100000:0.0:1787826302.943688:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:24703:x1874663346182400:12345-0@lo:101:mdt00_007.0 00010000:00010000:0.0:1787826302.943736:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826302.943773:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a4dc lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a4d5 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.943816:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a4dc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a4d5 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.943968:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a4dc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a4d5 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826302.944048:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5300) 00010000:00010000:0.0:1787826302.944052:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826302.944057:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826302.944202:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182400:12345-0@lo:101:mdt00_007.0 Request processed in 519us (7535us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.944209:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23552 00000100:00100000:3.0:1787826302.946456:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182400:0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826302.946495:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.946512:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826302.946550:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826302.953786:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182528:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826302.954010:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.954067:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f43c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826302.954454:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346182528 00000100:00100000:3.0:1787826302.954465:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346182528 00000100:00100000:3.0:1787826302.954541:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23553 00000100:00100000:3.0:1787826302.954553:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182528:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826302.954679:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182528:12345-0@lo:1000:mdt00_007.0 Request processed in 128us (670us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.954703:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23553 00000100:00100000:2.0:1787826302.955833:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182528:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826302.955939:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182656:0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826302.956008:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826302.956047:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.956166:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346182656 00000100:00100000:0.0:1787826302.956173:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346182656 00000100:00100000:0.0:1787826302.956232:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23554 00000100:00100000:0.0:1787826302.956243:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182656:12345-0@lo:1000:mdt00_007.0 00000100:00100000:0.0:1787826302.956500:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182656:12345-0@lo:1000:mdt00_007.0 Request processed in 258us (490us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826302.956507:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23554 00000100:00100000:3.0:1787826302.956595:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182656:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826302.956699:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182784:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826302.956739:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826302.956768:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aac00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.957073:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346182784 00000100:00100000:0.0:1787826302.957079:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346182784 00000100:00100000:0.0:1787826302.957163:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23555 00000100:00100000:0.0:1787826302.957171:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182784:12345-0@lo:1000:mdt00_007.0 00000100:00100000:0.0:1787826302.957261:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182784:12345-0@lo:1000:mdt00_007.0 Request processed in 90us (520us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826302.957268:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23555 00000100:00100000:3.0:1787826302.957506:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182784:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826302.957576:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182912:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826302.957605:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826302.957632:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aac00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.957790:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346182912 00000100:00100000:2.0:1787826302.957796:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346182912 00000100:00100000:2.0:1787826302.957838:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23556 00000100:00100000:2.0:1787826302.957846:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182912:12345-0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826302.957921:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346182912:12345-0@lo:1000:mdt00_007.0 Request processed in 76us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.957929:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23556 00000100:00100000:1.0:1787826302.958013:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346182912:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826302.958065:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346183040:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826302.958097:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.958152:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826302.958311:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346183040 00000100:00100000:2.0:1787826302.958316:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346183040 00000100:00100000:2.0:1787826302.958418:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23557 00000100:00100000:2.0:1787826302.958427:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346183040:12345-0@lo:1000:mdt00_007.0 00000100:00100000:2.0:1787826302.958502:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346183040:12345-0@lo:1000:mdt00_007.0 Request processed in 76us (403us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826302.958509:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23557 00000100:00100000:1.0:1787826302.958820:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346183040:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826302.958889:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346183168:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826302.958919:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.958946:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826302.959406:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346183168 00000100:00100000:0.0:1787826302.959414:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346183168 00000100:00100000:0.0:1787826302.959465:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23558 00000100:00100000:0.0:1787826302.959476:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346183168:12345-0@lo:1000:mdt00_007.0 00000100:00100000:0.0:1787826302.959560:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346183168:12345-0@lo:1000:mdt00_007.0 Request processed in 86us (640us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826302.959568:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23558 00000100:00100000:1.0:1787826302.959816:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346183168:0@lo:1000:mdt00_007.0 00000040:00080000:1.0:1787826302.980583:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1194, 2168 off1087360 00000040:00080000:1.0:1787826302.980750:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1512, 2368 off1528344 00000004:00080000:1.0:1787826302.980758:0:24703:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81180 with version 6815 00000004:00080000:0.0:1787826302.980862:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6815 rpc_version 6815 00000020:00100000:0.0:1787826302.980951:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826302.980966:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c95500 x1874663346183424/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826302.981006:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346183424:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826302.981145:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.981198:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826302.983098:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346183424 00000100:00100000:1.0:1787826302.983156:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346183424 00000100:00100000:1.0:1787826302.983222:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23559 00000100:00100000:1.0:1787826302.983233:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346183424:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826302.986994:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346183424:12345-0@lo:1000:osp_up0-1.0 Request processed in 3751us (5837us total) trans 8589957366 rc 0/0 00000100:00100000:1.0:1787826302.987010:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23559 00000100:00100000:2.0:1787826302.987367:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346183424:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826302.989767:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.989787:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.989801:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.989823:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.989830:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.989837:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.989875:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.989882:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.989892:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.989919:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346183552:0@lo:101:mdt00_007.0 00000100:00100000:1.0:1787826302.989989:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826302.990030:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784753bd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826302.991171:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346183552 00000100:00100000:3.0:1787826302.991194:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346183552 00000100:00100000:3.0:1787826302.991303:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23560 00000100:00100000:3.0:1787826302.991318:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:24703:x1874663346183552:12345-0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826302.991369:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826302.992211:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a4f8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a4f1 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.992327:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a4f8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a4f1 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.992346:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a4f8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a4f1 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.992356:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826302.992359:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826302.992364:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826302.992536:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:24703:x1874663346183552:12345-0@lo:101:mdt00_007.0 Request processed in 1221us (2546us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826302.992545:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23560 00000100:00100000:1.0:1787826302.993152:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346183552:0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826302.993177:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993190:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826302.993221:0:24703:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000777971c6 reply state 0000000025f293f3 transno 4294985378 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993236:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993244:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:1.0:1787826302.993246:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826302.993251:0:24703:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826302.993257:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993265:0:24703:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826302.993280:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00000004:00010000:1.0:1787826302.993291:0:24703:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000777971c6 reply state 0000000025f293f3 transno 4294985378 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993316:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993322:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:1.0:1787826302.993325:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826302.993326:0:24703:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787826302.993332:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4a:0x0]: returning size 4096 00000004:00080000:1.0:1787826302.993341:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826302.993351:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993358:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993365:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993388:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826302.993393:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826302.993397:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826302.993404:0:24703:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826302.993494:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224505:x1874663332572800:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 73745us (74075us total) trans 4294985378 rc 0/0 00000100:00100000:1.0:1787826302.993501:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68363 00010000:00010000:3.0:1787826302.993513:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826302.993521:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.993530:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826302.993536:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826302.993544:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.993575:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826302.993580:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826302.993656:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346183680:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826302.993716:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826302.993824:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346183680 00000100:00100000:0.0:1787826302.993832:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346183680 00010000:00010000:3.0:1787826303.000751:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.000771:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4a4c7 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a4ce expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.002180:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826303.002186:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826303.002189:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826303.003214:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.003307:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332573184 00000100:00100000:3.0:1787826303.003317:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332573184 00000100:00100000:3.0:1787826303.003380:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68364 00000100:00100000:3.0:1787826303.003393:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224494:x1874663332573184:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826303.003417:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826303.003438:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a4ff lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012f1 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.003508:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.003523:0:24702:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.003532:0:24702:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00100000:3.0:1787826303.003556:0:24702:0:(ldlm_lockd.c:2635:ldlm_cancel_hpreq_lock_match()) @@@ Prio raised by lock 0xe21309aa81f4a4ce req@ffff9f7851241180 x1874663346183680/t0(0) o103->lustre-MDT0001-mdtlov_UUID@0@lo:658/0 lens 328/0 e 0 to 0 dl 1787826313 ref 1 fl New:/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787826303.003587:0:24702:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826303.003602:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:24702:1874663346183808:0@lo:104: 00000100:00100000:3.0:1787826303.003668:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826303.003726:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826303.003753:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826303.003765:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346183808 00000100:00100000:3.0:1787826303.003768:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346183808 00000100:00100000:3.0:1787826303.003801:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7846 00000100:00100000:3.0:1787826303.003808:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79469a7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:24702:x1874663346183808:12345-0@lo:104: 00010000:00010000:3.0:1787826303.003816:0:74129:0:(ldlm_lockd.c:2448:ldlm_callback_handler()) callback on lock 0xe21309aa81f4a4c7 - lock disappeared 00010000:00010000:3.0:1787826303.003855:0:74129:0:(ldlm_lockd.c:2345:ldlm_callback_errmsg()) @@@ Operate with invalid parameter, NID=12345-0@lo lock=0xe21309aa81f4a4c7: rc = 0 req@ffff9f79469a7850 x1874663346183808/t0(0) o104->lustre-MDT0001-mdtlov_UUID@0@lo:659/0 lens 328/224 e 0 to 0 dl 1787826314 ref 1 fl Interpret:/0/0 rc -22/-22 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826303.003865:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79469a7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:24702:x1874663346183808:12345-0@lo:104: Request processed in 59us (199us total) trans 0 rc -22/-22 00000100:00100000:3.0:1787826303.003872:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7846 00010000:00010000:3.0:1787826303.004092:0:24702:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 0@lo) returned -22 from blocking AST (req@00000000bef3987a x1874663346183808) - normal race ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004143:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a4ce lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a4c7 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826303.004158:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:24702:1874663346183808:0@lo:104: 00010000:00010000:3.0:1787826303.004178:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826303.004183:0:24702:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004191:0:24702:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004198:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004213:0:24702:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004222:0:24702:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004227:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826303.004283:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826303.004288:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826303.004308:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004332:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a4ff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012f1 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004346:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012f1 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.004353:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826303.004356:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826303.004359:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826303.004405:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224494:x1874663332573184:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1015us (1198us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826303.004412:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68364 00000100:00080000:2.0:1787826303.004728:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826303.007204:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21917 00000100:00100000:0.0:1787826303.007230:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346183680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826303.007251:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826303.007256:0:125406:0:(ldlm_lockd.c:1754:ldlm_request_cancel()) ### server-side cancel handler stale lock (cookie e21309aa81f4a4ce) 00010000:00010000:0.0:1787826303.007259:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826303.007337:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346183680:12345-0@lo:103:ldlm_bl.0 Request processed in 118us (13630us total) trans 0 rc 116/116 00000100:00100000:0.0:1787826303.007344:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21917 00000100:00100000:2.0:1787826303.007453:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346183680:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826303.012279:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.012427:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332573312 00000100:00100000:3.0:1787826303.012450:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332573312 00000100:00100000:3.0:1787826303.012511:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68365 00000100:00100000:3.0:1787826303.012522:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224494:x1874663332573312:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826303.012541:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826303.012563:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a50d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012f8 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.012628:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.012647:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.012657:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.012668:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.012680:0:11629:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a4dc lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a4d5 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.012689:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.012739:0:11629:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a4dc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a4d5 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826303.012757:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346183936:0@lo:104: 00000100:00100000:3.0:1787826303.012806:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826303.012845:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826303.012862:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346183936 00000100:00100000:3.0:1787826303.012865:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346183936 00000100:00100000:3.0:1787826303.012894:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7847 00000100:00100000:3.0:1787826303.012901:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346183936:12345-0@lo:104: 00010000:00010000:3.0:1787826303.012914:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.012958:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826303.012964:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346183936:12345-0@lo:104: Request processed in 64us (159us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826303.012972:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7847 00010000:00010000:3.0:1787826303.012992:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826303.012998:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.013007:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826303.013013:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826303.013023:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.013049:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.013057:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826303.013079:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.013088:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4a4d5 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a4dc expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.013131:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826303.013134:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826303.013136:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826303.013148:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b662a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346184064:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826303.013178:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826303.014559:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346183936:0@lo:104: 00010000:00010000:3.0:1787826303.014593:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826303.014807:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346184064 00000100:00100000:1.0:1787826303.014814:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346184064 00010000:00010000:1.0:1787826303.014851:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a4dc lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a4d5 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826303.015038:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21918 00000100:00100000:1.0:1787826303.015048:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346184064:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826303.015062:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787826303.020777:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957366 is committed 00010000:00010000:1.0:1787826303.021040:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a4dc lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a4d5 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.021059:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5300) 00010000:00010000:1.0:1787826303.021065:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.021074:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.021081:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.021609:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826303.021726:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.021735:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826303.021871:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:11629:1874663346184192:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787826303.022076:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3652:x1874663346184064:12345-0@lo:103:ldlm_bl.0 Request processed in 7028us (8895us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826303.022087:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21918 00000100:00100000:1.0:1787826303.022247:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b662a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346184064:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826303.022553:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826303.022588:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aac00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826303.022668:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346184192 00000100:00100000:1.0:1787826303.022675:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346184192 00000100:00100000:1.0:1787826303.022728:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23561 00000100:00100000:1.0:1787826303.022738:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:11629:x1874663346184192:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787826303.023130:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:11629:x1874663346184192:12345-0@lo:1000:mdt00_004.0 Request processed in 392us (576us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826303.023150:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23561 00000100:00100000:3.0:1787826303.027080:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:11629:1874663346184192:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787826303.031352:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a522 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031377:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a522 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031389:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a522 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826303.031423:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a522 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031441:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031448:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031455:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031464:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031475:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826303.031480:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031488:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031500:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826303.031503:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826303.031507:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031513:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a514 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031536:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031542:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031547:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031552:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826303.031555:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031560:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031570:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5300) 00010000:00010000:2.0:1787826303.031573:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826303.031576:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031581:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a51b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031625:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a50d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012f8 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031640:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a522 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012f8 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.031648:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784abfe600) 00010000:00010000:2.0:1787826303.031651:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826303.031653:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826303.031782:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224494:x1874663332573312:12345-192.168.203.56@tcp:101:bash.0 Request processed in 19260us (19512us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826303.031790:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68365 00000100:00100000:0.0:1787826303.050271:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.050399:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332573440 00000100:00100000:3.0:1787826303.050410:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332573440 00000100:00100000:3.0:1787826303.050497:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68366 00000100:00100000:3.0:1787826303.050510:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224494:x1874663332573440:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826303.050537:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826303.050559:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4a529 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608012ff expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.050619:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a530 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.050633:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a530 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.050648:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a530 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826303.050761:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4a:0x0]: returning size 4096 00000004:00200000:3.0:1787826303.050766:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826303.050805:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4a529 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ff expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.050824:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a530 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ff expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.050832:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:3.0:1787826303.050835:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826303.050839:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826303.050899:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224494:x1874663332573440:12345-192.168.203.56@tcp:101:bash.0 Request processed in 393us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826303.050906:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68366 00000100:00080000:2.0:1787826303.050996:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826303.063461:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.063583:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332573568 00000100:00100000:3.0:1787826303.063593:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332573568 00000100:00100000:3.0:1787826303.063657:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68367 00000100:00100000:3.0:1787826303.063668:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224494:x1874663332573568:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826303.063688:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826303.063719:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4a537 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801306 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.063780:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4a53e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.063798:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4a53e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.063809:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4a53e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826303.063853:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x193:0x0]: returning size 4096 00000004:00200000:3.0:1787826303.063858:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826303.063912:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4a537 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801306 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.063931:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4a53e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801306 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.063940:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:3.0:1787826303.063943:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826303.063947:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826303.064016:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224494:x1874663332573568:12345-192.168.203.56@tcp:101:bash.0 Request processed in 350us (566us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826303.064024:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68367 00000100:00100000:1.0:1787826303.067602:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.067691:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332573696 00000100:00100000:3.0:1787826303.067703:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332573696 00000100:00100000:3.0:1787826303.067778:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68368 00000100:00100000:3.0:1787826303.067789:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224494:x1874663332573696:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826303.067811:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826303.067832:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a545 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080130d expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.068403:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.068419:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a4f8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a4f1 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.068428:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 4/1,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.068462:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a4f8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a4f1 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826303.068479:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346184320:0@lo:104: 00000100:00100000:3.0:1787826303.068532:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826303.068636:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aac00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826303.072384:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346184320 00000100:00100000:2.0:1787826303.072401:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346184320 00000100:00100000:2.0:1787826303.072509:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7848 00000100:00100000:2.0:1787826303.072520:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346184320:12345-0@lo:104: 00010000:00010000:2.0:1787826303.072541:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.072665:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826303.072672:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346184320:12345-0@lo:104: Request processed in 155us (4139us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826303.072679:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7848 00000100:00100000:0.0:1787826303.072775:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346184320:0@lo:104: 00010000:00010000:0.0:1787826303.072809:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.076005:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826303.076022:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.076035:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826303.076045:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826303.076056:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.076136:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.076144:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826303.076170:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.076179:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4a4f1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a4f8 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.076205:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826303.076207:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826303.076208:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826303.076219:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826303.076242:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346184448:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826303.076320:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826303.076374:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346184448 00000100:00100000:1.0:1787826303.076380:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346184448 00010000:00010000:1.0:1787826303.076412:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a4f8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a4f1 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826303.076444:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21919 00000100:00100000:1.0:1787826303.076453:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd80050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3652:x1874663346184448:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826303.076465:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826303.077581:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a4f8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a4f1 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.077600:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:1.0:1787826303.077605:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.077614:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.077622:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.077648:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826303.077719:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd80050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346184448:12345-0@lo:103:ldlm_bl.0 Request processed in 1265us (1398us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826303.077727:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21919 00010000:00010000:1.0:1787826303.077787:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.077796:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826303.077858:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x191:0x0]: returning size 4096 00000004:00200000:1.0:1787826303.077863:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826303.077908:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4a545 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080130d expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.077923:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080130d expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.077929:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:1.0:1787826303.077931:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826303.077933:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:1.0:1787826303.077992:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224494:x1874663332573696:12345-192.168.203.56@tcp:101:bash.0 Request processed in 10205us (10400us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826303.078000:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68368 00000100:00100000:1.0:1787826303.078054:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346184448:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826303.085868:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.086001:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332573952 00000100:00100000:3.0:1787826303.086011:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332573952 00000100:00100000:3.0:1787826303.086075:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68369 00000100:00100000:3.0:1787826303.086087:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224494:x1874663332573952:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826303.086127:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826303.086139:0:35093:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826303.086168:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4a53e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801306 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.086189:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:3.0:1787826303.086194:0:35093:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826303.086208:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4a553 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801314 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.086339:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 3/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.086352:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.086361:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.086420:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.086558:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.086567:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826303.086701:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346184576:0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826303.086772:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826303.086927:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826303.087665:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346184576 00000100:00100000:2.0:1787826303.087675:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346184576 00000100:00100000:2.0:1787826303.087747:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23562 00000100:00100000:2.0:1787826303.087757:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35093:x1874663346184576:12345-0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826303.087862:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35093:x1874663346184576:12345-0@lo:1000:mdt00_010.0 Request processed in 107us (1089us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826303.087870:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23562 00000100:00100000:0.0:1787826303.087970:0:35093:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957366 gen 2 00000004:00080000:0.0:1787826303.087986:0:35093:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957366 committed transno 8589957366 00000100:00100000:0.0:1787826303.088018:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346184576:0@lo:1000:mdt00_010.0 00000004:00080000:0.0:1787826303.088353:0:35093:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4cc:0x0], index 0: 0 00010000:00010000:0.0:1787826303.088518:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a568 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088533:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a568 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088545:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a568 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088579:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088587:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088594:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088600:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088606:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826303.088611:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088620:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088633:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:0.0:1787826303.088636:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826303.088641:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 3/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088647:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4a55a lrc: 0/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826303.088662:0:35093:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b5b0ea97 reply state 0000000046c87c29 transno 4294985379 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088673:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088680:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f7961d81900) 00010000:00010000:0.0:1787826303.088683:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826303.088686:0:35093:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826303.088731:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4a553 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801314 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088747:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a568 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801314 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.088755:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:0.0:1787826303.088758:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826303.088761:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826303.088819:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224494:x1874663332573952:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2734us (2962us total) trans 4294985379 rc 0/0 00000100:00100000:0.0:1787826303.088827:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68369 00000100:00100000:1.0:1787826303.096056:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.096148:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332574080 00000100:00100000:3.0:1787826303.096159:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332574080 00000100:00080000:2.0:1787826303.096367:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826303.096425:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30811 00000100:00100000:0.0:1787826303.096439:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:224494:x1874663332574080:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826303.096458:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826303.096483:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080131b expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826303.096510:0:24218:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4cc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787826303.096531:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=8115s v=(0 0) 00010000:00010000:0.0:1787826303.096535:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826303.096546:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.096564:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.096579:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826303.096646:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224494:x1874663332574080:12345-192.168.203.56@tcp:101:bash.0 Request processed in 211us (598us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826303.096653:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30811 00000100:00100000:1.0:1787826303.109584:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.109767:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332574336 00000100:00100000:3.0:1787826303.109777:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332574336 00000100:00100000:3.0:1787826303.109838:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9331 00000100:00100000:3.0:1787826303.109850:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224494:x1874663332574336:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826303.110609:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224494:x1874663332574336:12345-192.168.203.56@tcp:35:bash.0 Request processed in 755us (1029us total) trans 4294985380 rc 0/0 00000100:00100000:2.0:1787826303.110620:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9331 00000100:00100000:1.0:1787826303.119279:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826303.119479:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332574592 00000100:00100000:0.0:1787826303.119490:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332574592 00000100:00100000:0.0:1787826303.119562:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68370 00000100:00100000:0.0:1787826303.119573:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224494:x1874663332574592:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826303.119596:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826303.119600:0:6527:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826303.119630:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a54c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080130d expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.119652:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:0.0:1787826303.119663:0:6527:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826303.119679:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a57d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801322 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.119820:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.119833:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.119842:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.119901:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.119909:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.119917:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826303.120449:0:6527:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x51f:0x0], index 1: 0 00010000:00010000:0.0:1787826303.120595:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a592 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120606:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a592 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120613:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a592 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120646:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120654:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120660:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120667:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120674:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826303.120678:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120685:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120696:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:0.0:1787826303.120699:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826303.120702:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120719:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a584 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826303.120732:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002243401b reply state 0000000022cf7343 transno 8589957367 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120743:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120749:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x4ec94fd6 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826303.120752:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826303.120754:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826303.120783:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a57d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801322 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120803:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a592 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801322 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826303.120811:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f784b4a5000) 00010000:00010000:0.0:1787826303.120814:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826303.120816:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826303.120894:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224494:x1874663332574592:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1323us (1625us total) trans 8589957367 rc 0/0 00000100:00100000:0.0:1787826303.120903:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68370 00000100:00100000:1.0:1787826303.129414:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826303.129510:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332574720 00000100:00100000:2.0:1787826303.129522:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332574720 00000100:00100000:2.0:1787826303.129591:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30812 00000100:00100000:2.0:1787826303.129603:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224494:x1874663332574720:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826303.129623:0:8444:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826303.129646:0:8444:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801329 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826303.129674:0:8444:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x51f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826303.129696:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51f:0x0].0x0 since=8115s v=(0 0) 00010000:00010000:2.0:1787826303.129701:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826303.129721:0:8444:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 7 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.129740:0:8444:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 7 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826303.129755:0:8444:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:2.0:1787826303.129821:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224494:x1874663332574720:12345-192.168.203.56@tcp:101:bash.0 Request processed in 221us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826303.129829:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30812 00000100:00080000:0.0:1787826303.129906:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826303.145016:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826303.145289:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332574976 00000100:00100000:0.0:1787826303.145299:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332574976 00000100:00100000:0.0:1787826303.145364:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9332 00000100:00100000:0.0:1787826303.145375:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224494:x1874663332574976:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826303.145680:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224494:x1874663332574976:12345-192.168.203.56@tcp:35:bash.0 Request processed in 305us (673us total) trans 8589957368 rc 0/0 00000100:00100000:0.0:1787826303.145689:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9332 00010000:00010000:2.0:1787826303.149005:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826303.171569:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826303.172079:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332575104 00000100:00100000:1.0:1787826303.172089:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332575104 00000100:00100000:1.0:1787826303.172427:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11065 00000100:00100000:1.0:1787826303.172440:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:15673:x1874663332575104:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826303.172458:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826303.172485:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a568 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801314 expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.172506:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:1.0:1787826303.172526:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a530 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012ff expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.172536:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:1.0:1787826303.172544:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a2f2 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e48:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801288 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826303.172551:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e48:0x0].0x0 (ffff9f79460a2b00) 00010000:00010000:1.0:1787826303.172557:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826303.172634:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:15673:x1874663332575104:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 197us (1076us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826303.172642:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11065 00000100:00100000:2.0:1787826303.175610:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826303.175702:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332575232 00000100:00100000:3.0:1787826303.175725:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332575232 00000100:00100000:3.0:1787826303.175792:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11066 00000100:00100000:3.0:1787826303.175803:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:6166:x1874663332575232:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826303.175818:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826303.175843:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a592 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801322 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.175874:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f784b4a5000) 00010000:00010000:3.0:1787826303.175893:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a506 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012f1 expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.175903:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826303.175923:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a522 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608012f8 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.175934:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784abfe600) 00010000:00010000:3.0:1787826303.175942:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826303.176002:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:6166:x1874663332575232:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 201us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826303.176010:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11066 00010000:00010000:2.0:1787826304.180533:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826304.224342:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826304.224398:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826304.224430:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826304.224532:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332575360 00000100:00100000:1.0:1787826304.224543:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332575360 00000100:00100000:1.0:1787826304.224600:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2218 00000100:00100000:1.0:1787826304.224611:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf42450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332575360:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826304.224712:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf42450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332575360:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 102us (379us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826304.224720:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2218 00000100:00100000:0.0:1787826304.225004:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332575616 00000100:00100000:0.0:1787826304.225009:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332575616 00000100:00100000:0.0:1787826304.225084:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14274 00000100:00100000:0.0:1787826304.225093:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2395:x1874663332575616:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826304.225157:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2395:x1874663332575616:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 66us (758us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826304.225164:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14274 00000100:00080000:1.0:1787826304.225690:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826304.225803:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826304.225837:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826304.225879:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332576128 00000100:00100000:3.0:1787826304.225886:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332576128 00000100:00100000:3.0:1787826304.225931:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14275 00000100:00100000:3.0:1787826304.225941:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2395:x1874663332576128:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826304.226002:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2395:x1874663332576128:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 63us (202us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826304.226008:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14275 00000100:00100000:1.0:1787826304.228503:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826304.228544:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826304.228583:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332575488 00000100:00100000:0.0:1787826304.228592:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332575488 00000100:00100000:0.0:1787826304.228657:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14276 00000100:00100000:0.0:1787826304.228668:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332575488:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826304.228764:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332575488:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 97us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826304.228773:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14276 00000100:00100000:2.0:1787826304.231613:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332575872 00000100:00100000:2.0:1787826304.231629:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332575872 00000100:00100000:2.0:1787826304.231784:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332576384 00000100:00100000:2.0:1787826304.231788:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332576384 00000100:00100000:2.0:1787826304.231825:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8798 00000100:00100000:2.0:1787826304.231835:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332575872:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826304.231905:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332575872:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 71us (7474us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826304.231912:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8798 00000100:00100000:2.0:1787826304.231931:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8799 00000100:00100000:2.0:1787826304.231937:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332576384:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826304.231960:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332576384:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (6123us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826304.232199:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8799 00000100:00080000:0.0:1787826304.232293:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787826304.232299:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826304.233175:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332575744 00000100:00100000:2.0:1787826304.233184:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332575744 00000100:00100000:1.0:1787826304.233307:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826304.233330:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332576000 00000100:00100000:1.0:1787826304.233335:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332576000 00000100:00100000:1.0:1787826304.233372:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14277 00000100:00100000:1.0:1787826304.233381:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332576000:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826304.233442:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332576000:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 62us (137us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826304.233449:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14277 00000100:00100000:1.0:1787826304.233509:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826304.233757:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332576256 00000100:00100000:1.0:1787826304.233762:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332576256 00000100:00100000:1.0:1787826304.233791:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8800 00000100:00100000:1.0:1787826304.233798:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332576256:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826304.233832:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332576256:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 34us (324us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826304.233838:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8800 00000100:00100000:2.0:1787826304.233953:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8801 00000100:00100000:2.0:1787826304.233962:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332575744:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826304.234000:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332575744:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 40us (5455us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826304.234007:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8801 00000100:00080000:0.0:1787826304.234203:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787826304.430188:0:165662:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:1.0:1787826304.430231:0:165662:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000100:00100000:1.0:1787826304.734801:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826304.734855:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826304.735640:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332576640 00000100:00100000:2.0:1787826304.735653:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332576640 00000100:00100000:1.0:1787826304.735792:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30813 00000100:00100000:1.0:1787826304.735807:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332576640:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:1.0:1787826304.735935:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332576640:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 132us (1146us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826304.735943:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30813 00000100:00100000:3.0:1787826304.739052:0:38010:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332576512 00000100:00100000:3.0:1787826304.739065:0:38010:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332576512 00002000:00100000:3.0:1787826304.739143:0:38010:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_013: refresh rw locks for [0x280000400:0x4cc:0x0] (0->5) req@ffff9f7851884e00 x1874663332576512/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:660/0 lens 496/0 e 0 to 0 dl 1787826315 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826304.739154:0:38010:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7851884e00 with x1874663332576512 ext(0->5) 00002000:00010000:3.0:1787826304.739161:0:38010:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7851884e00 00000100:00100000:3.0:1787826304.739208:0:38010:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27642 00000100:00100000:3.0:1787826304.739219:0:38010:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332576512:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826304.743518:0:38010:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332576512:12345-192.168.203.56@tcp:4:bash.0 Request processed in 4289us (8651us total) trans 4294973845 rc 0/0 00000100:00100000:3.0:1787826304.743533:0:38010:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27642 00002000:00100000:3.0:1787826304.743549:0:38010:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_013: refresh rw locks for [0x280000400:0x4cc:0x0] (0->5) req@ffff9f7851884e00 x1874663332576512/t4294973845(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:660/0 lens 496/448 e 0 to 0 dl 1787826315 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826304.743558:0:38010:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7851884e00 with x1874663332576512 ext(0->5) 00002000:00010000:3.0:1787826304.743564:0:38010:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7851884e00 00000100:00100000:1.0:1787826304.768836:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826304.768888:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826304.768988:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332576896 00000100:00100000:0.0:1787826304.768998:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332576896 00000100:00100000:0.0:1787826304.769054:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30814 00000100:00100000:0.0:1787826304.769064:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332576896:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:0.0:1787826304.769189:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332576896:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 127us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826304.769196:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30814 00000100:00100000:3.0:1787826304.770226:0:24220:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332576768 00000100:00100000:3.0:1787826304.770237:0:24220:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332576768 00002000:00100000:3.0:1787826304.770293:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x51f:0x0] (0->5) req@ffff9f7851887480 x1874663332576768/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:660/0 lens 496/0 e 0 to 0 dl 1787826315 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826304.770303:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7851887480 with x1874663332576768 ext(0->5) 00002000:00010000:3.0:1787826304.770310:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7851887480 00000100:00100000:3.0:1787826304.770344:0:24220:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27643 00000100:00100000:3.0:1787826304.770352:0:24220:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332576768:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826304.772609:0:24220:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332576768:12345-192.168.203.56@tcp:4:bash.0 Request processed in 2246us (3771us total) trans 4294976153 rc 0/0 00000100:00100000:3.0:1787826304.772624:0:24220:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27643 00002000:00100000:3.0:1787826304.772638:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000401:0x51f:0x0] (0->5) req@ffff9f7851887480 x1874663332576768/t4294976153(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:660/0 lens 496/448 e 0 to 0 dl 1787826315 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826304.772648:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7851887480 with x1874663332576768 ext(0->5) 00002000:00010000:3.0:1787826304.772654:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7851887480 00000100:00100000:0.0:1787826305.005630:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346184704:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.005731:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.005783:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346185216:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.005816:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.005836:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346185728:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.005860:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.005878:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346186240:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.005900:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.005933:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1787826305.005939:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346184832:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006036:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.006057:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346185344:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006079:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.006095:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d27480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346185856:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006144:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.006188:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346184704 00000100:00100000:0.0:1787826305.006196:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346184704 00000100:00100000:0.0:1787826305.006262:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2219 00000100:00100000:0.0:1787826305.006272:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf40c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346184704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006328:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf40c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346184704:12345-0@lo:400:kworker.0 Request processed in 56us (597us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.006336:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2219 00000100:00100000:0.0:1787826305.006380:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346185216 00000100:00100000:0.0:1787826305.006384:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346185216 00000100:00100000:0.0:1787826305.006425:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346186240 00000100:00100000:0.0:1787826305.006429:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346186240 00000100:00100000:0.0:1787826305.006449:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14278 00000100:00100000:0.0:1787826305.006456:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346185216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006491:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346185216:12345-0@lo:400:kworker.0 Request processed in 35us (674us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.006498:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14278 00000100:00100000:0.0:1787826305.006511:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14279 00000100:00100000:0.0:1787826305.006518:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346186240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006545:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346186240:12345-0@lo:400:kworker.0 Request processed in 28us (644us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.006552:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14279 00000100:00100000:0.0:1787826305.006573:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346185728 00000100:00100000:0.0:1787826305.006577:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346185728 00000100:00100000:0.0:1787826305.006605:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346185344 00000100:00100000:0.0:1787826305.006608:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346185344 00000100:00100000:0.0:1787826305.006630:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8802 00000100:00100000:0.0:1787826305.006637:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346185728:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006667:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346185728:12345-0@lo:400:kworker.0 Request processed in 31us (807us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.006674:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8802 00000100:00100000:0.0:1787826305.006685:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8803 00000100:00100000:0.0:1787826305.006702:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346185344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006732:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346185344:12345-0@lo:400:kworker.0 Request processed in 41us (652us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.006737:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8803 00000100:00100000:0.0:1787826305.006758:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346184832 00000100:00100000:0.0:1787826305.006762:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346184832 00000100:00100000:0.0:1787826305.006789:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346185856 00000100:00100000:0.0:1787826305.006792:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346185856 00000100:00100000:0.0:1787826305.006809:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23563 00000100:00100000:0.0:1787826305.006815:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346184832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006848:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346184832:12345-0@lo:400:kworker.0 Request processed in 33us (811us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.006855:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23563 00000100:00100000:0.0:1787826305.006866:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23564 00000100:00100000:0.0:1787826305.006872:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346185856:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.006896:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346185856:12345-0@lo:400:kworker.0 Request processed in 24us (751us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.006902:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23564 00000100:00100000:3.0:1787826305.007160:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346185088:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.008651:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346184832:0@lo:400:kworker.0 00000100:00080000:1.0:1787826305.011444:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826305.011457:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826305.011461:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826305.011788:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346184960:0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.011982:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826305.012026:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346185600:0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.012048:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826305.012079:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346185088 00000100:00100000:3.0:1787826305.012088:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346185088 00000100:00100000:3.0:1787826305.012163:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346185600 00000100:00100000:3.0:1787826305.012166:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346185600 00000100:00100000:3.0:1787826305.012272:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14280 00000100:00100000:3.0:1787826305.012282:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346185088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.012342:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346185088:12345-0@lo:400:kworker.0 Request processed in 61us (360us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826305.012349:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14280 00000100:00100000:3.0:1787826305.012370:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14281 00000100:00100000:3.0:1787826305.012377:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346185600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.012407:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346185600:12345-0@lo:400:kworker.0 Request processed in 32us (358us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826305.012413:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14281 00000100:00080000:3.0:1787826305.012428:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826305.012431:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826305.012463:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346185600:0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.012472:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346186112:0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.012500:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826305.014703:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346185088:0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.014808:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346186112 00000100:00100000:3.0:1787826305.014819:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346186112 00000100:00100000:3.0:1787826305.014895:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8804 00000100:00100000:3.0:1787826305.014906:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346186112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.015024:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346186112:12345-0@lo:400:kworker.0 Request processed in 119us (2522us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826305.015032:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8804 00000100:00080000:3.0:1787826305.015056:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826305.015094:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976151 gen 2 00000004:00080000:3.0:1787826305.015102:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f78435b3480, transno 4294976151 00000100:00100000:3.0:1787826305.015135:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346186112:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.019377:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346185344:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.019410:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d27480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346185856:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.019423:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346184704:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.019437:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346185216:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.019452:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346185728:0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.019465:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346186240:0@lo:400:kworker.0 00000100:00100000:1.0:1787826305.020580:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826305.020669:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346185472:0@lo:400:kworker.0 00000100:00100000:1.0:1787826305.020716:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826305.020821:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d27b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346185984:0@lo:400:kworker.0 00000100:00100000:1.0:1787826305.020851:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826305.020975:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346185984 00000100:00100000:2.0:1787826305.020985:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346185984 00000100:00100000:2.0:1787826305.021070:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8805 00000100:00100000:2.0:1787826305.021081:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346185984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826305.021170:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346185984:12345-0@lo:400:kworker.0 Request processed in 90us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826305.021178:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8805 00000100:00080000:1.0:1787826305.021319:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826305.021353:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d27b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346185984:0@lo:400:kworker.0 00000100:00100000:3.0:1787826305.021432:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346184960 00000100:00100000:3.0:1787826305.021437:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346184960 00000100:00100000:0.0:1787826305.021529:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346185472 00000100:00100000:0.0:1787826305.021533:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346185472 00000100:00100000:0.0:1787826305.021580:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14282 00000100:00100000:0.0:1787826305.021587:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346185472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826305.021630:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346185472:12345-0@lo:400:kworker.0 Request processed in 44us (913us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.021637:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14282 00000100:00100000:0.0:1787826305.021667:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346185472:0@lo:400:kworker.0 00000100:00080000:1.0:1787826305.021729:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826305.021816:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14283 00000100:00100000:2.0:1787826305.021823:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346184960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826305.021864:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346184960:12345-0@lo:400:kworker.0 Request processed in 42us (1293us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826305.021870:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14283 00000100:00100000:1.0:1787826305.021922:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346184960:0@lo:400:kworker.0 00010000:00010000:3.0:1787826305.205272:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1787826305.519706:0:9993:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8117731597155ns, next at 8118731593832ns, rc = 0 00000100:00100000:3.0:1787826305.519793:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826305.519817:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346186368:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826305.519914:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826305.520142:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346186368 00000100:00100000:1.0:1787826305.520151:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346186368 00000100:00100000:3.0:1787826305.520210:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6219 00000100:00100000:3.0:1787826305.520233:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346186368:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826305.520259:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914711040 avail 3683188736 00000100:00100000:3.0:1787826305.520316:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346186368:12345-0@lo:13:osp-pre-1-1.0 Request processed in 96us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826305.520322:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6219 00000100:00100000:2.0:1787826305.520460:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346186368:0@lo:13:osp-pre-1-1.0 00000001:00080000:2.0:1787826305.523321:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973845 is committed 00000001:00080000:2.0:1787826305.523337:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973844 is committed 00000001:00080000:2.0:1787826305.523345:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973843 is committed 00010000:00010000:3.0:1787826305.524044:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a42d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826305.524063:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a42d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826305.524139:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a42d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826305.524146:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826305.524175:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf600/0xe21309aa81f4a426 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a42d expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826305.524195:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985380 is committed 00000020:00080000:3.0:1787826305.524209:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7844e81c00 refcount 2 committed 0 result 0 batchid 3409 00000020:00080000:3.0:1787826305.524213:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952e3c400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e81180 00000020:00080000:3.0:1787826305.524217:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1512 00000020:00080000:3.0:1787826305.524220:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952e3c900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852be1600 00000020:00080000:3.0:1787826305.524222:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1194 00000001:00080000:3.0:1787826305.524236:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985377 is committed 00000020:00080000:3.0:1787826305.524241:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78500aa600 refcount 2 committed 0 result 0 batchid 3408 00000020:00080000:3.0:1787826305.524244:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7944e37000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78500aae40 00000020:00080000:3.0:1787826305.524247:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1511 00000020:00080000:3.0:1787826305.524249:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7944e37b00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852be0e00 00000020:00080000:3.0:1787826305.524252:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1193 00000004:00080000:3.0:1787826305.524262:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 401 committed 00000020:00080000:3.0:1787826305.524274:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78524059c0 refcount 2 committed 0 result 0 batchid 3407 00000020:00080000:3.0:1787826305.524277:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852405cc0 00000020:00080000:3.0:1787826305.524280:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1510 00000020:00080000:3.0:1787826305.524282:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7847aa3400 00000020:00080000:3.0:1787826305.524284:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1192 00000020:00080000:3.0:1787826305.524292:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7844e81780 refcount 2 committed 0 result 0 batchid 3406 00000020:00080000:3.0:1787826305.524295:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e81c00 00000020:00080000:3.0:1787826305.524298:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1509 00000020:00080000:3.0:1787826305.524300:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839800 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7846231800 00000020:00080000:3.0:1787826305.524302:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1191 00000001:00080000:3.0:1787826305.524308:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985371 is committed 00000020:00080000:3.0:1787826305.524320:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3405 batchid updated 00000100:00100000:3.0:1787826305.524332:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826305.524354:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346186496:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787826305.524425:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826305.524918:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.524929:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.524936:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.524942:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826305.524947:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.524953:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.524967:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:2.0:1787826305.524970:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826305.524975:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.524981:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a4e3 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.524998:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525005:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525011:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525016:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826305.525019:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525025:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525033:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:2.0:1787826305.525036:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826305.525039:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525044:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a4ea lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525060:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525066:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525071:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525076:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826305.525079:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525084:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525124:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f7961d81900) 00010000:00010000:2.0:1787826305.525127:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826305.525131:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826305.525137:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a561 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787826305.525223:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3409 committed batchid 3405 00000004:00080000:3.0:1787826305.525335:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78500aa300 with version 6816 00000020:00080000:3.0:1787826305.525354:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3402 cancel update log [0x200000401:0x3:0x0].1505: rc = 0 00000020:00080000:3.0:1787826305.525400:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3402 cancel update log [0x240000400:0x3:0x0].1186: rc = 0 00000004:00080000:3.0:1787826305.525425:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78500aa780 with version 6817 00000020:00080000:3.0:1787826305.525428:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3403 cancel update log [0x200000401:0x3:0x0].1506: rc = 0 00000020:00080000:3.0:1787826305.525451:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3403 cancel update log [0x240000400:0x3:0x0].1187: rc = 0 00000004:00080000:2.0:1787826305.525617:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6816 rpc_version 6816 00000020:00100000:2.0:1787826305.525646:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826305.525657:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7851242d80 x1874663346186752/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826305.525679:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346186752:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826305.525751:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826305.525793:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826305.525982:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346186752 00000100:00100000:1.0:1787826305.525991:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346186752 00000004:00080000:3.0:1787826305.526049:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78500aac00 with version 6818 00000020:00080000:3.0:1787826305.526054:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3404 cancel update log [0x200000401:0x3:0x0].1507: rc = 0 00000100:00100000:1.0:1787826305.526084:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23565 00000020:00080000:3.0:1787826305.526086:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3404 cancel update log [0x240000400:0x3:0x0].1188: rc = 0 00000004:00080000:3.0:1787826305.526192:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846153540 with version 6819 00000020:00080000:3.0:1787826305.526196:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3405 cancel update log [0x200000401:0x3:0x0].1508: rc = 0 00000020:00080000:3.0:1787826305.526219:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3405 cancel update log [0x240000400:0x3:0x0].1190: rc = 0 00000020:00080000:3.0:1787826305.526227:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3405 00000100:00100000:3.0:1787826305.526308:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346186496 00000100:00100000:3.0:1787826305.526312:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346186496 00000100:00100000:3.0:1787826305.526347:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11067 00000100:00100000:3.0:1787826305.526353:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346186496:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787826305.526365:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826305.526385:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4a42d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x190:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a426 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826305.526399:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x190:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:3.0:1787826305.526407:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826305.526447:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346186496:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 93us (2021us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826305.529022:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11067 00000100:00100000:3.0:1787826305.529102:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346186880:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1787826305.529197:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826305.529263:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346186496:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826305.530932:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346186752:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826305.531233:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346186752:12345-0@lo:1000:osp_up0-1.0 Request processed in 5136us (5478us total) trans 8589957369 rc 0/0 00000100:00100000:1.0:1787826305.531242:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23565 00000100:00100000:1.0:1787826305.531325:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346186880 00000100:00100000:1.0:1787826305.531332:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346186880 00000100:00100000:1.0:1787826305.531374:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30815 00000100:00100000:1.0:1787826305.531384:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346186880:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:1.0:1787826305.531404:0:17826:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1225 count 1 00010000:00010000:1.0:1787826305.531430:0:17826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826305.531452:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 8 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826305.531460:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4c9:0x0].0x0 since=9s v=(0 0) 00010000:00010000:1.0:1787826305.531464:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826305.531469:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826305.531501:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 8 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826305.531516:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 8 pid: 24218 timeout: 8217 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826305.531529:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f2b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346187008:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826305.531600:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826305.534359:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346186752:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826305.534387:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6817 rpc_version 6817 00000020:00100000:2.0:1787826305.534412:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826305.534419:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78433efb80 x1874663346187264/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826305.534451:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346187264:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826305.534525:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826305.534563:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826305.537970:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346187264 00000100:00100000:0.0:1787826305.537986:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346187264 00000100:00100000:0.0:1787826305.538047:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23566 00000100:00100000:0.0:1787826305.538057:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346187264:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826305.538298:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346187264:12345-0@lo:1000:osp_up0-1.0 Request processed in 242us (3771us total) trans 8589957370 rc 0/0 00000100:00100000:0.0:1787826305.538306:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23566 00000100:00100000:0.0:1787826305.538376:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346187264:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826305.538392:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6818 rpc_version 6818 00000020:00100000:0.0:1787826305.538414:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826305.538421:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7967913100 x1874663346187520/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826305.538443:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967913100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346187520:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826305.538483:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.538506:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78474d7e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826305.538522:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346187520 00000100:00100000:0.0:1787826305.538526:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346187520 00000100:00100000:0.0:1787826305.538552:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23567 00000100:00100000:0.0:1787826305.538559:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967910a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346187520:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826305.538638:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967910a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346187520:12345-0@lo:1000:osp_up0-1.0 Request processed in 79us (155us total) trans 8589957371 rc 0/0 00000100:00100000:0.0:1787826305.538644:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23567 00000100:00100000:0.0:1787826305.538677:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967913100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346187520:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826305.538686:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6819 rpc_version 6819 00000020:00100000:0.0:1787826305.538709:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826305.538713:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7967911880 x1874663346187776/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826305.538727:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967911880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346187776:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826305.538752:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826305.538767:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784aa28cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826305.538778:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346187776 00000100:00100000:0.0:1787826305.538782:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346187776 00000100:00100000:0.0:1787826305.538802:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23568 00000100:00100000:0.0:1787826305.538808:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346187776:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826305.538873:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346187776:12345-0@lo:1000:osp_up0-1.0 Request processed in 65us (119us total) trans 8589957372 rc 0/0 00000100:00100000:0.0:1787826305.538880:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23568 00000100:00100000:0.0:1787826305.538910:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967911880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346187776:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826305.546865:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f2b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346187008:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826305.546901:0:17826:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826305.547754:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826305.547901:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332577024 00000100:00100000:0.0:1787826305.547910:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332577024 00010000:00010000:0.0:1787826305.547967:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 9 pid: 24218 timeout: 8217 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826305.547987:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826305.548007:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21920 00000100:00100000:0.0:1787826305.548018:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332577024:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826305.548033:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826305.548039:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 9 pid: 24218 timeout: 8217 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826305.548051:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 9 pid: 24218 timeout: 8217 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826305.548076:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4a24a lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801249 expref: 9 pid: 24218 timeout: 8217 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826305.548094:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826305.548125:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826305.548131:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826305.548142:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826305.548170:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826305.548184:0:17826:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826305.548196:0:17826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826305.548206:0:17826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826305.548223:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332577024:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 206us (473us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826305.548232:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21920 00010000:00010000:3.0:1787826305.548606:0:17826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826305.548633:0:17826:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826305.548817:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826305.548825:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826305.548826:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346186880:12345-0@lo:6:osp-syn-1-1.0 Request processed in 17439us (19625us total) trans 4294976154 rc 0/0 00000100:00100000:3.0:1787826305.548845:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30815 00010000:00010000:0.0:1787826305.548854:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826305.548860:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826305.548869:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826305.548884:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826305.548892:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4a5a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826305.548901:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7854627480/2, rc 0, transno 4294976154 00000100:00100000:3.0:1787826305.548910:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346186880:0@lo:6:osp-syn-1-1.0 00010000:00010000:0.0:1787826305.548912:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826305.548916:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826305.548918:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826306.028514:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826306.028619:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967913480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346187904:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826306.028722:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826306.028910:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346187904 00000100:00100000:0.0:1787826306.028921:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346187904 00000100:00100000:0.0:1787826306.028992:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6220 00000100:00100000:0.0:1787826306.029004:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346187904:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787826306.029041:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914706944 avail 3683184640 00000100:00100000:0.0:1787826306.029126:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346187904:12345-0@lo:13:osp-pre-1-0.0 Request processed in 100us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826306.029134:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6220 00000100:00100000:0.0:1787826306.029197:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967913480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346187904:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826306.030469:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346188032:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826306.030833:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967913480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346188160:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826306.030904:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826306.030955:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826306.030962:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f2a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346188288:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826306.030988:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826306.031073:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826306.031339:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346188416:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826306.031370:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826306.031835:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346188160 00000100:00100000:0.0:1787826306.031846:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346188160 00000100:00100000:0.0:1787826306.031908:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346188416 00000100:00100000:0.0:1787826306.031911:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346188416 00000100:00100000:0.0:1787826306.031936:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23569 00000100:00100000:0.0:1787826306.031947:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346188160:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1787826306.031987:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443405824 avail 1297166336 00000100:00100000:0.0:1787826306.032044:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346188160:12345-0@lo:41:osp-pre-1-0.0 Request processed in 98us (1140us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826306.032052:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23569 00000100:00100000:0.0:1787826306.032072:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23570 00000100:00100000:0.0:1787826306.032078:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346188416:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787826306.032102:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442803712 avail 1296564224 00000100:00100000:0.0:1787826306.032167:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346188416:12345-0@lo:41:osp-pre-0-1.0 Request processed in 89us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826306.032173:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23570 00000100:00100000:0.0:1787826306.032224:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346188416:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826306.032385:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985371 gen 2 00000004:00080000:3.0:1787826306.032391:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985370 committed transno 4294985371 00000020:00080000:3.0:1787826306.032400:0:3652:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7844e81900 refcount 2 committed 0 result 0 batchid 4294979080 00000020:00080000:3.0:1787826306.032404:0:3652:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839200 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e81000 00000020:00080000:3.0:1787826306.032408:0:3652:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].936 00000020:00080000:3.0:1787826306.032410:0:3652:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a9f1800 00000020:00080000:3.0:1787826306.032413:0:3652:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].933 00000004:00080000:3.0:1787826306.032445:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985371 committed transno 4294985371 00000100:00100000:3.0:1787826306.032467:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967913480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346188160:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826306.032511:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346188288 00000100:00100000:3.0:1787826306.032516:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346188288 00000100:00100000:3.0:1787826306.032673:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346188032 00000100:00100000:3.0:1787826306.032677:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346188032 00000020:00080000:0.0:1787826306.032736:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979080 committed batchid 4294979079 00000020:00080000:0.0:1787826306.032813:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979079 00000100:00100000:1.0:1787826306.034071:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6221 00000100:00100000:1.0:1787826306.034096:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346188288:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826306.034149:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662454784 00000100:00100000:1.0:1787826306.034422:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346188288:12345-0@lo:13:osp-pre-0-1.0 Request processed in 336us (3430us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826306.034441:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6221 00000100:00100000:1.0:1787826306.034470:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6222 00000100:00100000:1.0:1787826306.034480:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967912680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346188032:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826306.034495:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662454784 00000100:00100000:1.0:1787826306.034532:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967912680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346188032:12345-0@lo:13:osp-pre-0-0.0 Request processed in 55us (3459us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826306.034539:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6222 00000100:00100000:1.0:1787826306.034598:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973844 gen 2 00000004:00080000:1.0:1787826306.034736:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784b750700, transno 4294973844 00000100:00100000:1.0:1787826306.034763:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346188032:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826306.037577:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973843 gen 2 00000004:00080000:2.0:1787826306.037592:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784836e300, transno 4294973843 00000100:00100000:2.0:1787826306.038074:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f2a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346188288:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1787826306.222996:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826306.951731:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826306.955640:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332577152 00000100:00100000:0.0:1787826306.955655:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332577152 00000100:00100000:0.0:1787826306.956372:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68371 00000100:00100000:0.0:1787826306.956387:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224675:x1874663332577152:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826306.956410:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826306.956433:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a5ae lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801330 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.956500:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.956515:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.956529:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826306.956589:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826306.956592:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826306.956612:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.956639:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a5ae lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.956655:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.956663:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826306.956666:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826306.956669:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826306.956752:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224675:x1874663332577152:12345-192.168.203.56@tcp:101:cat.0 Request processed in 368us (5032us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826306.956759:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68371 00000100:00080000:1.0:1787826306.962561:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826306.970704:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826306.971319:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332577280 00000100:00100000:1.0:1787826306.971331:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332577280 00000100:00100000:1.0:1787826306.971432:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68372 00000100:00100000:1.0:1787826306.971444:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967913480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224675:x1874663332577280:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826306.971466:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826306.971489:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a5bc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801337 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971555:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971573:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971584:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971599:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971607:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971613:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971700:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a5d1 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971708:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a5d1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971714:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a5d1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826306.971737:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a5d1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971753:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971759:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971769:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971775:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971781:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826306.971786:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971792:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971805:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826306.971808:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826306.971812:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971819:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4a5c3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971837:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971844:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971850:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971856:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826306.971859:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971865:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971871:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1500) 00010000:00010000:1.0:1787826306.971874:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826306.971876:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971882:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5ca lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971914:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a5bc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801337 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971928:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a5d1 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801337 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.971935:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:1.0:1787826306.971937:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826306.971940:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826306.972014:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967913480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224675:x1874663332577280:12345-192.168.203.56@tcp:101:cat.0 Request processed in 572us (1319us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826306.972022:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68372 00000100:00100000:2.0:1787826306.974435:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826306.977394:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332577408 00000100:00100000:0.0:1787826306.977410:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332577408 00000100:00100000:0.0:1787826306.977537:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68373 00000100:00100000:0.0:1787826306.977548:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224675:x1874663332577408:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826306.977568:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826306.977593:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a5d8 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080133e expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.977653:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a5df lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.977668:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a5df lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.977690:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a5df lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826306.977798:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4a:0x0]: returning size 4096 00000004:00200000:0.0:1787826306.977803:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826306.979950:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a5d8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080133e expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.979984:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a5df lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080133e expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826306.979994:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:0.0:1787826306.979998:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826306.980001:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826306.980081:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224675:x1874663332577408:12345-192.168.203.56@tcp:101:cat.0 Request processed in 2533us (5654us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826306.980090:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68373 00000100:00080000:0.0:1787826306.980259:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826306.988770:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826306.989083:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332577536 00000100:00100000:2.0:1787826306.989094:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332577536 00000100:00100000:2.0:1787826306.989175:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68374 00000100:00100000:2.0:1787826306.989185:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224675:x1874663332577536:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826306.989208:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826306.989299:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4a5e6 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801345 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826306.989361:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 3/1,0 mode: --/PR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826306.989374:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826306.989385:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826306.989429:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x193:0x0]: returning size 4096 00000004:00200000:2.0:1787826306.989433:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826306.989468:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4a5e6 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826306.989485:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826306.989494:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:2.0:1787826306.989498:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826306.989501:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826306.989557:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224675:x1874663332577536:12345-192.168.203.56@tcp:101:cat.0 Request processed in 373us (797us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826306.989565:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68374 00000100:00100000:0.0:1787826306.999322:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826306.999409:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332577664 00000100:00100000:1.0:1787826306.999419:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332577664 00000100:00100000:1.0:1787826306.999476:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68375 00000100:00100000:1.0:1787826306.999488:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224675:x1874663332577664:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826306.999509:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826306.999535:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a5f4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080134c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.999596:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.999609:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.999620:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826306.999663:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x191:0x0]: returning size 4096 00000004:00200000:1.0:1787826306.999668:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826306.999716:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a5f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.999744:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826306.999754:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:1.0:1787826306.999758:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826306.999761:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826306.999820:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224675:x1874663332577664:12345-192.168.203.56@tcp:101:cat.0 Request processed in 334us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826306.999827:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68375 00000100:00100000:0.0:1787826307.002139:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826307.002920:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332577920 00000100:00100000:1.0:1787826307.002931:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332577920 00000100:00100000:1.0:1787826307.002995:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68376 00000100:00100000:1.0:1787826307.003006:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224675:x1874663332577920:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826307.003024:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826307.003046:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a602 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801353 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.003142:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.003155:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.003167:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.003181:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.003189:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.003196:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00020000:1.0:1787826307.003332:0:11629:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00010000:00010000:2.0:1787826307.244348:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826307.464235:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826307.465442:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332578048 00000100:00100000:2.0:1787826307.465460:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332578048 00000100:00100000:2.0:1787826307.465577:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68377 00000100:00100000:2.0:1787826307.465591:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224677:x1874663332578048:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826307.465619:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826307.465647:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a617 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080135a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.465746:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.465765:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.465779:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826307.465841:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826307.465846:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826307.465886:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.465925:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a617 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.466230:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.466242:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826307.466245:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826307.466250:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826307.466336:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224677:x1874663332578048:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 747us (2114us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826307.466346:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68377 00000100:00080000:0.0:1787826307.466827:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826307.477010:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826307.477166:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332578176 00000100:00100000:1.0:1787826307.477179:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332578176 00000100:00100000:1.0:1787826307.477256:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68378 00000100:00100000:1.0:1787826307.477268:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224677:x1874663332578176:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826307.477295:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826307.477320:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a625 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801361 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477405:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477422:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477435:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477453:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477461:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477468:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477551:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477560:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477568:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826307.477595:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477614:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477622:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477628:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477635:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477643:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826307.477647:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477656:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477670:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826307.477684:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826307.477691:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477699:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a62c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477718:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477725:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477732:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477737:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826307.477740:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477747:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477755:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1300) 00010000:00010000:1.0:1787826307.477757:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826307.477761:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477767:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a633 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477807:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a625 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477823:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.477830:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:1.0:1787826307.477832:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826307.477835:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826307.477913:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224677:x1874663332578176:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 646us (920us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.477924:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68378 00000100:00100000:1.0:1787826307.481421:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826307.481769:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332578304 00000100:00100000:0.0:1787826307.481782:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332578304 00000100:00100000:0.0:1787826307.481940:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68379 00000100:00100000:0.0:1787826307.481963:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:224677:x1874663332578304:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826307.481991:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826307.482018:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a641 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801368 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.482094:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a648 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.482159:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a648 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.482173:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a648 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826307.482293:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4a:0x0]: returning size 4096 00000004:00200000:0.0:1787826307.482300:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826307.482351:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4a641 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801368 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.482379:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a648 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801368 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.482389:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:0.0:1787826307.482392:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826307.482397:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826307.482471:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224677:x1874663332578304:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 512us (1064us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826307.482480:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68379 00000100:00080000:2.0:1787826307.483318:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826307.489797:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826307.490060:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332578432 00000100:00100000:3.0:1787826307.490073:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332578432 00000100:00100000:3.0:1787826307.490184:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68380 00000100:00100000:3.0:1787826307.490607:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224677:x1874663332578432:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826307.490638:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826307.490664:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a64f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080136f expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.490768:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.490785:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.490798:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.490811:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.490820:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.490827:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.490958:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a664 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.491432:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a664 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.491444:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a664 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826307.491923:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db9:0x0]: returning size 6 00000004:00200000:3.0:1787826307.491929:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826307.492303:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a664 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492321:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a664 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492333:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492340:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492347:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492361:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492367:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826307.492372:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492381:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492399:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:3.0:1787826307.492402:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826307.492408:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492414:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a656 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492433:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492440:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492446:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492452:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826307.492455:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492594:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492606:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x4ec94fd6 (ffff9f784b4a5400) 00010000:00010000:3.0:1787826307.492608:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826307.492612:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492618:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a65d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492661:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4a64f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080136f expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492687:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a664 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080136f expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.492696:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:3.0:1787826307.492698:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826307.492700:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826307.492858:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224677:x1874663332578432:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2660us (3075us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826307.492876:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68380 00000100:00100000:3.0:1787826307.502797:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826307.503356:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332578560 00000100:00100000:2.0:1787826307.503368:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332578560 00000100:00100000:2.0:1787826307.503443:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68381 00000100:00100000:2.0:1787826307.503456:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224677:x1874663332578560:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826307.503494:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826307.503530:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4a66b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801376 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503607:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503624:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503637:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503652:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503660:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503666:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503775:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a680 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503784:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a680 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503791:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a680 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826307.503841:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4b:0x0]: returning size 6 00000004:00200000:2.0:1787826307.503846:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826307.503868:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a680 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503881:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a680 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503892:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503900:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503912:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503919:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503925:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826307.503929:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503937:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503966:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:2.0:1787826307.503970:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826307.503975:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.503982:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4a672 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504001:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504007:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504014:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504019:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826307.504023:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504029:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504037:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f7845cb1d00) 00010000:00010000:2.0:1787826307.504040:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826307.504044:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504050:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a679 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504078:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4a66b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801376 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504094:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a680 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801376 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.504102:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:2.0:1787826307.504161:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826307.504164:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826307.504377:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224677:x1874663332578560:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 922us (1593us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826307.504387:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68381 00000100:00100000:1.0:1787826307.509741:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826307.509802:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826307.509893:0:6541:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332578944 00000100:00100000:2.0:1787826307.509906:0:6541:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332578944 00000100:00100000:2.0:1787826307.509998:0:6541:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1228 00000100:00100000:2.0:1787826307.510012:0:6541:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224677:x1874663332578944:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:2.0:1787826307.510092:0:6541:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826307.510169:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4a680 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801376 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.510203:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:2.0:1787826307.510218:0:6541:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826307.510295:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afc000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346188544:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.510359:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.510428:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826307.510754:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332578688 00000100:00100000:0.0:1787826307.510759:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332578688 00000100:00100000:0.0:1787826307.510813:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11068 00000100:00100000:0.0:1787826307.510823:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:2393:x1874663332578688:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:0.0:1787826307.510835:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826307.510852:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4a664 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080136f expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.510868:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:0.0:1787826307.510877:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826307.510948:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2393:x1874663332578688:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 117us (1214us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826307.510957:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11068 00000100:00100000:3.0:1787826307.511237:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346188544 00000100:00100000:3.0:1787826307.511243:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346188544 00000100:00100000:3.0:1787826307.511280:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23571 00000100:00100000:3.0:1787826307.511287:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346188544:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.517528:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346188544:12345-0@lo:1000:mdt_io00_001.0 Request processed in 6220us (7147us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826307.517610:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23571 00000100:00100000:2.0:1787826307.518433:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afc000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346188544:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.518642:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afc000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346188672:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.518710:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.518843:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826307.520281:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346188672 00000100:00100000:1.0:1787826307.520295:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346188672 00000100:00100000:1.0:1787826307.520466:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23572 00000100:00100000:1.0:1787826307.520477:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346188672:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.520647:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346188672:12345-0@lo:1000:mdt_io00_001.0 Request processed in 171us (1936us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.520653:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23572 00000100:00100000:3.0:1787826307.520742:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afc000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346188672:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826307.520786:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00000100:00100000:3.0:1787826307.520869:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346188800:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.520913:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826307.521549:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826307.521747:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346188800 00000100:00100000:2.0:1787826307.521754:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346188800 00000100:00100000:2.0:1787826307.521804:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23573 00000100:00100000:2.0:1787826307.521814:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346188800:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.521920:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346188800:12345-0@lo:1000:mdt_io00_001.0 Request processed in 107us (1006us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826307.521927:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23573 00000100:00100000:2.0:1787826307.522291:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346188800:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826307.522342:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00000100:00100000:2.0:1787826307.522445:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845aff800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346188928:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.522482:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.522625:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826307.522642:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346188928 00000100:00100000:2.0:1787826307.522647:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346188928 00000100:00100000:2.0:1787826307.522672:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23574 00000100:00100000:2.0:1787826307.522690:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346188928:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.522751:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346188928:12345-0@lo:1000:mdt_io00_001.0 Request processed in 62us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826307.522757:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23574 00000100:00100000:2.0:1787826307.522783:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845aff800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346188928:0@lo:1000:mdt_io00_001.0 00000004:00020000:2.0:1787826307.522816:0:6541:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:2.0:1787826307.522919:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.522937:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00020000:3.0:1787826307.523499:0:11629:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4479 00010000:00010000:2.0:1787826307.523648:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.523700:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.523717:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 8 pid: 24703 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826307.523730:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845aff800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346189056:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826307.523806:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4000 going to sleep for 11 seconds 00010000:00010000:3.0:1787826307.524148:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524160:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524171:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826307.524184:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4b:0x0]: returning size 6 00010000:00010000:3.0:1787826307.524344:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524352:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524432:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524441:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524452:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826307.524457:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524464:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524478:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:3.0:1787826307.524481:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826307.524484:0:11629:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524492:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.524499:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4a609 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.525095:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.525724:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.525732:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.525738:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826307.525742:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.525754:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.525766:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f7845cb1d00) 00010000:00010000:3.0:1787826307.525769:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826307.525773:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.525779:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a610 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.528004:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4a602 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.528038:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.528047:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f7843326700) 00010000:00010000:3.0:1787826307.528050:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826307.528053:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826307.529787:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224675:x1874663332577920:12345-192.168.203.56@tcp:101:cat.0 Request processed in 526770us (527670us total) trans 4294985381 rc 0/0 00000100:00100000:3.0:1787826307.529800:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68376 00000100:00100000:1.0:1787826307.530135:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826307.530265:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845aff800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346189056:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826307.530289:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.532656:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332579072 00000100:00100000:0.0:1787826307.532671:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332579072 00010000:00010000:0.0:1787826307.532734:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 9 pid: 24703 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.532778:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21921 00000100:00100000:0.0:1787826307.532788:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332579072:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826307.532804:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826307.532811:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 9 pid: 24703 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.532825:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 9 pid: 24703 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.532847:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a5ed lrc: 0/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801345 expref: 9 pid: 24703 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.532866:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:0.0:1787826307.532871:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.532891:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.532899:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.532929:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826307.532983:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332579072:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 196us (2849us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826307.532991:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21921 00010000:00010000:0.0:1787826307.534195:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.534211:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.534242:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.534349:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.534360:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.534407:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.534414:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.534426:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.534448:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346189184:0@lo:101:mdt_io00_001.0 00000100:00100000:0.0:1787826307.534504:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826307.534600:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd1480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826307.540234:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826307.541194:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332579200 00000100:00100000:0.0:1787826307.541207:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332579200 00000100:00080000:2.0:1787826307.541276:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826307.541299:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30816 00000100:00100000:0.0:1787826307.541311:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332579200:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826307.541332:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826307.541353:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4a6aa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080137d expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.541416:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346189312:192.168.203.56@tcp:106: 00000100:00100000:0.0:1787826307.541513:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd1180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826307.543694:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346189184 00000100:00100000:1.0:1787826307.543713:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346189184 00000100:00100000:1.0:1787826307.543798:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23575 00000100:00100000:1.0:1787826307.543809:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346189184:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826307.543843:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826307.543868:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a6a3 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.543902:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.543913:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a6a3 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.543961:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.543976:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 8 pid: 24702 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826307.543991:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346189440:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826307.544086:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1a80 going to sleep for 11 seconds 00002000:00010000:0.0:1787826307.545827:0:8759:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x4cc:0x0] updating lvb size: 0 -> 6 00002000:00010000:0.0:1787826307.545836:0:8759:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x4cc:0x0] updating lvb mtime: 0 -> 1787826302 00002000:00010000:0.0:1787826307.545840:0:8759:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x4cc:0x0] updating lvb atime: 0 -> 1787826302 00002000:00010000:0.0:1787826307.545842:0:8759:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x4cc:0x0] updating lvb ctime: 0 -> 1787826302 00002000:00010000:0.0:1787826307.545844:0:8759:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x4cc:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:0.0:1787826307.545867:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346189312:192.168.203.56@tcp:106: 00010000:00010000:0.0:1787826307.545915:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4a6aa lrc: 1/0,0 mode: --/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080137d expref: 6 pid: 8759 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826307.545923:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346189440:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826307.545938:0:8759:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4a6aa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080137d expref: 6 pid: 8759 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.545944:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6a3 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.545949:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00010000:00010000:1.0:1787826307.545953:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:1.0:1787826307.545955:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826307.545960:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6a3 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.545967:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:0.0:1787826307.545998:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332579200:12345-192.168.203.56@tcp:101:cat.0 Request processed in 4690us (5773us total) trans 0 rc 301/301 00000100:00100000:0.0:1787826307.546007:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30816 00000100:00100000:1.0:1787826307.546045:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6541:x1874663346189184:12345-0@lo:101:mdt_io00_001.0 Request processed in 2238us (11540us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.546055:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23575 00000100:00100000:3.0:1787826307.546184:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346189184:0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826307.546200:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.546497:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826307.546772:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826307.546989:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332579328 00000100:00100000:0.0:1787826307.546998:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332579328 00010000:00010000:0.0:1787826307.547032:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 9 pid: 24702 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.547075:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21922 00000100:00100000:0.0:1787826307.547085:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd80c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332579328:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826307.547099:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826307.547120:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 9 pid: 24702 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.547133:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 9 pid: 24702 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.547152:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a5fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080134c expref: 9 pid: 24702 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.547166:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:0.0:1787826307.547172:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6a3 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.547182:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a6a3 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.547191:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a6a3 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.547223:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6a3 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.547238:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346189568:0@lo:105: 00000100:00100000:0.0:1787826307.547442:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826307.547488:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd1180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826307.547541:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346189568 00000100:00100000:2.0:1787826307.547546:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346189568 00000100:00100000:2.0:1787826307.547586:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7849 00000100:00100000:2.0:1787826307.547595:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346189568:12345-0@lo:105: 00010000:00010000:2.0:1787826307.547606:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.547616:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.547625:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.547633:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.547639:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.547646:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.547660:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43200) 00000100:00100000:2.0:1787826307.547715:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346189568:12345-0@lo:105: Request processed in 121us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826307.547722:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7849 00000100:00100000:0.0:1787826307.547824:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346189568:0@lo:105: 00010000:00010000:0.0:1787826307.547840:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826307.548344:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd80c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332579328:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1256us (1570us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826307.548352:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21922 00010000:00010000:3.0:1787826307.550041:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.550059:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826307.550092:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00010000:00010000:3.0:1787826307.550483:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.550493:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.551040:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826307.551351:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346189696:0@lo:101:mdt_io00_001.0 00000100:00100000:3.0:1787826307.551429:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826307.551525:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826307.554128:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346189696 00000100:00100000:1.0:1787826307.554142:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346189696 00000100:00100000:1.0:1787826307.554206:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23576 00000100:00100000:1.0:1787826307.554357:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6541:x1874663346189696:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826307.554449:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826307.554471:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 2/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a6b8 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.554502:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 2/0,1 mode: TXN/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.554924:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 4/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a6b8 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.554951:0:6532:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00080000:00080000:1.0:1787826307.554989:0:6532:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00000100:00100000:2.0:1787826307.555033:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:1.0:1787826307.555041:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 3/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6b8 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.555051:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x4ec94fd6 (ffff9f784b4a5400) 00010000:00010000:1.0:1787826307.555054:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826307.555058:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 3/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6b8 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.555066:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826307.555341:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:6541:x1874663346189696:12345-0@lo:101:mdt_io00_001.0 Request processed in 987us (3909us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.555350:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23576 00000100:00100000:3.0:1787826307.555420:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346189696:0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826307.555444:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.555453:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.560425:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332579456 00000100:00100000:0.0:1787826307.560442:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332579456 00000100:00100000:0.0:1787826307.560514:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30817 00000100:00100000:0.0:1787826307.560526:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224675:x1874663332579456:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826307.560546:0:35352:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826307.560566:0:35352:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801384 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.560598:0:35352:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.560607:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4cc:0x0].0x0 since=4s v=(0 0) 00010000:00010000:0.0:1787826307.560612:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826307.560617:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 4/0,0 mode: --/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 6 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.560653:0:35352:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.560668:0:35352:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 5 pid: 24218 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.560698:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346189824:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826307.560795:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd1240 going to sleep for 11 seconds 00000020:00080000:3.0:1787826307.562394:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979080 batchid updated 00000001:00080000:3.0:1787826307.562419:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957372 is committed 00000001:00080000:3.0:1787826307.562452:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957368 is committed 00000020:00080000:0.0:1787826307.562623:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979080 00000004:00080000:0.0:1787826307.562702:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7845dd10c0 with version 1822 00000020:00080000:0.0:1787826307.562719:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979080 cancel update log [0x240000404:0x2:0x0].936: rc = 0 00000020:00080000:0.0:1787826307.562858:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979080 cancel update log [0x200000bd0:0x2:0x0].933: rc = 0 00000020:00080000:0.0:1787826307.562866:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979080 00000004:00080000:3.0:1787826307.562972:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1822 rpc_version 1822 00000020:00100000:3.0:1787826307.563003:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826307.563013:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c96300 x1874663346190080/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826307.563035:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346190080:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787826307.563036:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.563046:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.563053:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.563058:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826307.563063:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.563072:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.563088:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x4ec94fd6 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826307.563092:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826307.563096:0:3639:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 2/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6b8 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.563144:0:3639:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a6b8 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.563153:0:3639:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a6b8 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.563179:0:3639:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6b8 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826307.563183:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.563189:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3639:1874663346190208:0@lo:105: 00000100:00100000:3.0:1787826307.563209:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826307.563216:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.563265:0:3639:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826307.563392:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346190208 00000100:00100000:1.0:1787826307.563400:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346190208 00000100:00100000:1.0:1787826307.563451:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7850 00000100:00100000:1.0:1787826307.563461:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:3639:x1874663346190208:12345-0@lo:105: 00010000:00010000:1.0:1787826307.563473:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.563482:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.563491:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 5/0,1 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.563499:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 5/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.563507:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.563514:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.563539:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69400) 00000100:00100000:1.0:1787826307.563583:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:3639:x1874663346190208:12345-0@lo:105: Request processed in 124us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.563590:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7850 00000100:00100000:1.0:1787826307.563628:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3639:1874663346190208:0@lo:105: 00010000:00010000:3.0:1787826307.563629:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.563638:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.563646:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.563652:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4a58b lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826307.563657:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00000100:00100000:3.0:1787826307.563710:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190336:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.563739:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826307.563763:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826307.563866:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346190080 00000100:00100000:1.0:1787826307.563926:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346190080 00000100:00100000:1.0:1787826307.563964:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346190336 00000100:00100000:1.0:1787826307.563967:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346190336 00000100:00100000:1.0:1787826307.563987:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23577 00000100:00100000:1.0:1787826307.563994:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf88c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:30909:x1874663346190080:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826307.564176:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf88c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346190080:12345-0@lo:1000:osp_up1-0.0 Request processed in 181us (991us total) trans 4294985382 rc 0/0 00000100:00100000:1.0:1787826307.564183:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23577 00000100:00100000:1.0:1787826307.564413:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23578 00000100:00100000:1.0:1787826307.564422:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190336:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.564476:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190336:12345-0@lo:1000:mdt_io00_001.0 Request processed in 55us (734us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.564483:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23578 00000100:00100000:1.0:1787826307.564526:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346190080:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826307.564795:0:6541:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957372 gen 2 00000004:00080000:3.0:1787826307.564802:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957369 committed transno 8589957372 00000004:00080000:3.0:1787826307.564816:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957370 committed transno 8589957372 00000004:00080000:3.0:1787826307.564825:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957371 committed transno 8589957372 00000004:00080000:3.0:1787826307.564847:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957372 committed transno 8589957372 00000100:00100000:3.0:1787826307.564863:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190336:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.564917:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190464:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.564948:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826307.564971:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.564985:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346190464 00000100:00100000:3.0:1787826307.564990:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346190464 00000100:00100000:3.0:1787826307.565020:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23579 00000100:00100000:3.0:1787826307.565026:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190464:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.565101:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190464:12345-0@lo:1000:mdt_io00_001.0 Request processed in 74us (151us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826307.565127:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23579 00000100:00100000:3.0:1787826307.565152:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190464:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.565187:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190592:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.565209:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826307.565221:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.565231:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346190592 00000100:00100000:3.0:1787826307.565234:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346190592 00000100:00100000:3.0:1787826307.565251:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23580 00000100:00100000:3.0:1787826307.565256:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190592:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.565348:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190592:12345-0@lo:1000:mdt_io00_001.0 Request processed in 91us (137us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826307.565354:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23580 00000100:00100000:3.0:1787826307.565378:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190592:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.565434:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190720:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.565456:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826307.566050:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826307.567483:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346189824:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826307.567530:0:35352:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 3/0,0 mode: --/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 6 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.567558:0:35352:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826307.567718:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224675:x1874663332579456:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7106us (12599us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826307.567728:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30817 00000100:00100000:1.0:1787826307.568054:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346190720 00000100:00100000:1.0:1787826307.568064:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346190720 00000100:00100000:1.0:1787826307.568120:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23581 00000100:00100000:1.0:1787826307.568130:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190720:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.568229:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190720:12345-0@lo:1000:mdt_io00_001.0 Request processed in 101us (2772us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.568237:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23581 00000100:00100000:2.0:1787826307.568644:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190720:0@lo:1000:mdt_io00_001.0 00010000:00010000:2.0:1787826307.568857:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.568873:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826307.568884:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826307.568909:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98dc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190848:0@lo:101:mdt_io00_001.0 00000100:00100000:2.0:1787826307.568953:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.568988:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.571201:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346190848 00000100:00100000:3.0:1787826307.571978:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346190848 00000100:00100000:3.0:1787826307.572053:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23582 00000100:00100000:3.0:1787826307.572074:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346190848:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826307.572095:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826307.572143:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a6d4 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a6cd expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.572176:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a6d4 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a6cd expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.572191:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a6d4 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6cd expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.572200:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f79460a2100) 00010000:00010000:3.0:1787826307.572203:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1787826307.572207:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826307.572294:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346190848:12345-0@lo:101:mdt_io00_001.0 Request processed in 231us (3339us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826307.572301:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23582 00000100:00100000:0.0:1787826307.572392:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98dc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190848:0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826307.572413:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6d4 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.572425:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6d4 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826307.572449:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000403:0x1db9:0x0] 00000100:00100000:0.0:1787826307.572516:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190976:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826307.572555:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826307.572582:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826307.572664:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346190976 00000100:00100000:1.0:1787826307.572670:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346190976 00000100:00100000:1.0:1787826307.572715:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23583 00000100:00100000:1.0:1787826307.572723:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346190976:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.572812:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346190976:12345-0@lo:1000:mdt_io00_001.0 Request processed in 89us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.572819:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23583 00000100:00100000:2.0:1787826307.572918:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826307.572988:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332579584 00000100:00100000:0.0:1787826307.572992:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332579584 00010000:00010000:0.0:1787826307.573035:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 6 pid: 24218 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.573067:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21923 00000100:00100000:0.0:1787826307.573074:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332579584:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826307.573084:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826307.573124:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 6 pid: 24218 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.573137:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 6 pid: 24218 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.573160:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4a576 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080131b expref: 6 pid: 24218 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.573175:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826307.573179:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826307.573187:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.573196:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.573227:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826307.573237:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6513:1874663346191104:192.168.203.56@tcp:105: 00000100:00100000:0.0:1787826307.573416:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.573513:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346190976:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826307.573554:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00010000:00010000:3.0:1787826307.573592:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826307.573599:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787826307.573605:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.573614:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.573638:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.573651:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 7 pid: 11629 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826307.573662:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346191232:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826307.573714:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826307.576527:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6513:1874663346191104:192.168.203.56@tcp:105: 00010000:00010000:0.0:1787826307.576568:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826307.576639:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332579584:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3563us (3720us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826307.576649:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21923 00000100:00100000:2.0:1787826307.579221:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826307.579318:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826307.579401:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346191232:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826307.579430:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332579840 00010000:00010000:3.0:1787826307.579445:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826307.579447:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332579840 00010000:00010000:1.0:1787826307.579514:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 8 pid: 11629 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826307.579568:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21924 00000100:00100000:1.0:1787826307.579581:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332579840:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826307.579604:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826307.579612:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 8 pid: 11629 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.579633:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 8 pid: 11629 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.579655:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4a68e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801353 expref: 8 pid: 11629 timeout: 8219 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.579682:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f7843326700) 00010000:00010000:1.0:1787826307.579688:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.579697:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.579705:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.579738:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826307.579771:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.579783:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826307.579800:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332579840:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 220us (480us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.579809:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21924 00010000:00010000:3.0:1787826307.579875:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.579886:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x2 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.579900:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x2 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826307.579923:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191360:0@lo:101:mdt_io00_001.0 00000100:00100000:3.0:1787826307.580069:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826307.580139:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826307.580190:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346191360 00000100:00100000:1.0:1787826307.580196:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346191360 00000100:00100000:1.0:1787826307.580234:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23584 00000100:00100000:1.0:1787826307.580243:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346191360:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826307.580395:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826307.580418:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a6e9 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a6e2 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.580443:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a6e9 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a6e2 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.580455:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a6e9 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a6e2 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.580463:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:1.0:1787826307.580466:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826307.580468:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826307.582282:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191360:12345-0@lo:101:mdt_io00_001.0 Request processed in 2028us (2202us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.582296:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23584 00000100:00100000:0.0:1787826307.582472:0:24160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332579712 00000100:00100000:0.0:1787826307.582483:0:24160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332579712 00002000:00100000:0.0:1787826307.582539:0:24160:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_011: refresh rw locks for [0x280000400:0x4cc:0x0] (0->4095) req@ffff9f784d98c380 x1874663332579712/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:663/0 lens 488/0 e 0 to 0 dl 1787826318 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826307.582555:0:24160:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d98c380 with x1874663332579712 ext(0->4095) 00002000:00010000:0.0:1787826307.582562:0:24160:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784d98c380 00000100:00100000:0.0:1787826307.582588:0:24160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27644 00000100:00100000:0.0:1787826307.582597:0:24160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332579712:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:0.0:1787826307.584261:0:24160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332579712:12345-192.168.203.56@tcp:3:cat.0 Request processed in 1649us (5034us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787826307.584275:0:24160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27644 00002000:00100000:0.0:1787826307.584567:0:24160:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_011: refresh rw locks for [0x280000400:0x4cc:0x0] (0->4095) req@ffff9f784d98c380 x1874663332579712/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:663/0 lens 488/4536 e 0 to 0 dl 1787826318 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826307.584585:0:24160:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d98c380 with x1874663332579712 ext(0->4095) 00002000:00010000:0.0:1787826307.584591:0:24160:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784d98c380 00000100:00100000:3.0:1787826307.588488:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191360:0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826307.588522:0:6541:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 4/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.588538:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.588550:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826307.589980:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826307.590070:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332579968 00000100:00100000:1.0:1787826307.590080:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332579968 00000100:00100000:1.0:1787826307.590166:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68382 00000100:00100000:1.0:1787826307.590179:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224675:x1874663332579968:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826307.590205:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826307.590226:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a6f0 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080138b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.590297:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a6f0 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080138b expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.590312:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a6f0 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080138b expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.590331:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f7843326700) 00010000:00010000:1.0:1787826307.590334:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787826307.590338:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826307.590409:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224675:x1874663332579968:12345-192.168.203.56@tcp:101:cat.0 Request processed in 232us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.590415:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68382 00000100:00100000:1.0:1787826307.591524:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191488:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.591586:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826307.591621:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.591754:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346191488 00000100:00100000:3.0:1787826307.591761:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346191488 00000100:00100000:3.0:1787826307.591811:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23585 00000100:00100000:3.0:1787826307.591820:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191488:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.591894:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191488:12345-0@lo:1000:mdt_io00_001.0 Request processed in 75us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826307.591901:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23585 00000100:00100000:1.0:1787826307.592082:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191488:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.592154:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191616:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.592178:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826307.592191:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.592688:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346191616 00000100:00100000:3.0:1787826307.592695:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346191616 00000100:00100000:3.0:1787826307.592735:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23586 00000100:00100000:3.0:1787826307.592744:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191616:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.592848:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191616:12345-0@lo:1000:mdt_io00_001.0 Request processed in 104us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826307.592854:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23586 00000100:00100000:2.0:1787826307.592937:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191616:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826307.592976:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00000100:00100000:2.0:1787826307.593027:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191744:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.593059:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.593096:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.593379:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346191744 00000100:00100000:3.0:1787826307.593385:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346191744 00000100:00100000:3.0:1787826307.593419:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23587 00000100:00100000:3.0:1787826307.593428:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191744:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.593487:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191744:12345-0@lo:1000:mdt_io00_001.0 Request processed in 61us (426us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826307.593493:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23587 00000100:00100000:2.0:1787826307.593699:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191744:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.593738:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191872:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.593766:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.593790:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.593906:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346191872 00000100:00100000:3.0:1787826307.593910:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346191872 00000100:00100000:3.0:1787826307.593942:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23588 00000100:00100000:3.0:1787826307.593948:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191872:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826307.594011:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346191872:12345-0@lo:1000:mdt_io00_001.0 Request processed in 63us (244us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826307.594017:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23588 00000100:00100000:2.0:1787826307.597551:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346191872:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.597703:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346192000:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826307.597779:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.597819:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826307.598154:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346192000 00000100:00100000:1.0:1787826307.598163:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346192000 00000100:00100000:1.0:1787826307.598231:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23589 00000100:00100000:1.0:1787826307.598241:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346192000:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.598341:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346192000:12345-0@lo:1000:mdt_io00_001.0 Request processed in 102us (566us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.598349:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23589 00000100:00100000:0.0:1787826307.598385:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346192000:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826307.598724:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826307.598786:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332580224 00000100:00100000:2.0:1787826307.598791:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332580224 00000100:00100000:2.0:1787826307.598832:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9333 00000100:00100000:2.0:1787826307.598840:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224675:x1874663332580224:12345-192.168.203.56@tcp:35:cat.0 00000100:00100000:2.0:1787826307.599148:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224675:x1874663332580224:12345-192.168.203.56@tcp:35:cat.0 Request processed in 307us (427us total) trans 4294985383 rc 0/0 00000100:00100000:2.0:1787826307.599156:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9333 00000040:00080000:0.0:1787826307.599667:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].226, 64 off22656 00000004:00080000:0.0:1787826307.599693:0:6541:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8119811586876ns, next at 8120811583224ns, rc = 0 00000040:00080000:0.0:1787826307.599740:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1195, 904 off1088264 00000040:00080000:0.0:1787826307.599788:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1513, 984 off1529328 00000004:00080000:0.0:1787826307.599794:0:6541:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784f7d7900 with version 6820 00000004:00080000:2.0:1787826307.601187:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6820 rpc_version 6820 00000020:00100000:2.0:1787826307.601247:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826307.601258:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9fea00 x1874663346192256/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2584/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826307.601284:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346192256:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826307.601351:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826307.601394:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826307.601533:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346192256 00000100:00100000:3.0:1787826307.601541:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346192256 00000100:00100000:3.0:1787826307.601598:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23590 00000100:00100000:3.0:1787826307.601609:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346192256:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826307.601959:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346192256:12345-0@lo:1000:osp_up0-1.0 Request processed in 351us (606us total) trans 8589957373 rc 0/0 00000100:00100000:3.0:1787826307.601966:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23590 00000100:00100000:2.0:1787826307.602338:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346192256:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826307.602429:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602441:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602448:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602455:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826307.602460:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602468:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602484:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f7843326700) 00010000:00010000:1.0:1787826307.602487:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826307.602491:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602497:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4a6db lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826307.602515:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000403:0x1db9:0x0] 00010000:00010000:1.0:1787826307.602523:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6d4 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602532:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6d4 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602544:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602550:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602555:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602559:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826307.602562:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602567:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602575:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:1.0:1787826307.602577:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826307.602580:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602585:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4a687 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826307.602605:0:6541:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000501507d8 reply state 00000000df4038d5 transno 4294985384 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602623:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602630:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f784b4a5900) 00010000:00010000:1.0:1787826307.602632:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826307.602635:0:6541:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:1.0:1787826307.602638:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00010000:00010000:1.0:1787826307.602643:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602650:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602656:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602689:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826307.602695:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00010000:00010000:1.0:1787826307.602700:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602707:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602716:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602724:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602731:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602753:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1787826307.602772:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602783:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602791:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602797:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602803:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602808:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.602822:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1787826307.602881:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787826307.602889:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826307.602900:0:6541:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224677:x1874663332578944:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 92886us (93092us total) trans 4294985384 rc 0/0 00010000:00010000:0.0:1787826307.602902:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826307.602910:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826307.602912:0:6541:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1228 00000004:00010000:0.0:1787826307.602923:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.602956:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.602963:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826307.602992:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.602999:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4a6a3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6b1 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.603016:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826307.603020:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.603026:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826307.603030:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826307.603038:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.603056:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826307.603063:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.603067:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.603072:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826307.603079:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.603080:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826307.603090:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.603100:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 1/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.603129:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f7843326700) 00010000:00010000:0.0:1787826307.603130:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4a6e2 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6e9 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.603136:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787826307.603141:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826307.603148:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4a6f7 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e4b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.603160:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826307.603161:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826307.603165:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826307.603166:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826307.603170:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826307.603171:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826307.603180:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826307.603194:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346192384:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826307.603265:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826307.603331:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346192384 00000100:00100000:1.0:1787826307.603338:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346192384 00000100:00100000:1.0:1787826307.603379:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11069 00000100:00100000:1.0:1787826307.603388:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346192384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826307.603400:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826307.603417:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a6b1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a6a3 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826307.603427:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:1.0:1787826307.603434:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826307.603792:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826307.603804:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346192512:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826307.603837:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826307.604202:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346192512 00000100:00100000:0.0:1787826307.604208:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346192512 00000100:00100000:0.0:1787826307.604531:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11070 00000100:00100000:0.0:1787826307.604541:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346192512:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826307.604552:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826307.604568:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a6e9 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a6e2 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826307.604587:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:0.0:1787826307.604595:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826307.609598:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346192512:12345-0@lo:103:ldlm_bl.0 Request processed in 5045us (5746us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826307.609614:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11070 00000100:00100000:2.0:1787826307.609644:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346192512:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826307.611901:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346192384:12345-0@lo:103:ldlm_bl.0 Request processed in 8504us (8640us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826307.611915:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11069 00000100:00100000:0.0:1787826307.617339:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346192384:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826308.268317:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826308.631491:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826308.631608:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332580352 00000100:00100000:3.0:1787826308.631618:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332580352 00000100:00100000:3.0:1787826308.631710:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68383 00000100:00100000:3.0:1787826308.631725:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224713:x1874663332580352:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826308.631753:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826308.631780:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a6fe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801392 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826308.631969:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1787826308.632051:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a705 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.632071:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a705 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.632082:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a705 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826308.636878:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4a:0x0]: returning size 4096 00000004:00200000:2.0:1787826308.636890:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787826308.636928:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a705 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.636972:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4a6fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801392 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.637001:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a705 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801392 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.637011:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:2.0:1787826308.637014:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826308.637018:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:2.0:1787826308.637094:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224713:x1874663332580352:12345-192.168.203.56@tcp:101:rm.0 Request processed in 5373us (5612us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.637121:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68383 00000100:00100000:1.0:1787826308.646844:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826308.646915:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332580480 00000100:00100000:2.0:1787826308.646926:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332580480 00000100:00100000:2.0:1787826308.646990:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68384 00000100:00100000:2.0:1787826308.647000:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224713:x1874663332580480:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826308.647020:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826308.647043:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4a70c lrc: 2/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801399 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.647303:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.647321:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.647332:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826308.647376:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x193:0x0]: returning size 4096 00000004:00200000:2.0:1787826308.647382:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826308.647415:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4a70c lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.647429:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.647437:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe300) 00010000:00010000:2.0:1787826308.647440:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826308.647444:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826308.647513:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224713:x1874663332580480:12345-192.168.203.56@tcp:101:rm.0 Request processed in 514us (679us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.647521:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68384 00000100:00100000:3.0:1787826308.662098:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826308.662418:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332580608 00000100:00100000:0.0:1787826308.662429:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332580608 00000100:00100000:0.0:1787826308.662507:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68385 00000100:00100000:0.0:1787826308.662518:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224713:x1874663332580608:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826308.662540:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826308.662566:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a71a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013a0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.662631:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.662645:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.662661:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826308.662725:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x191:0x0]: returning size 4096 00000004:00200000:0.0:1787826308.662729:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826308.662767:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a71a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x191:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.662783:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 9 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.662792:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:0.0:1787826308.662796:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826308.662799:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826308.662864:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224713:x1874663332580608:12345-192.168.203.56@tcp:101:rm.0 Request processed in 348us (775us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.662871:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68385 00000100:00080000:2.0:1787826308.663210:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826308.666424:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826308.666560:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332580864 00000100:00100000:0.0:1787826308.666571:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332580864 00000100:00100000:0.0:1787826308.666630:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68386 00000100:00100000:0.0:1787826308.666640:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224713:x1874663332580864:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826308.666663:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826308.666696:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4a728 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013a7 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826308.666848:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4a:0x0]: returning size 4096 00010000:00010000:0.0:1787826308.666941:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4a728 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013a7 expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.666957:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:0.0:1787826308.666960:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826308.666965:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4a728 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013a7 expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.666984:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826308.667047:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224713:x1874663332580864:12345-192.168.203.56@tcp:101:rm.0 Request processed in 409us (634us total) trans 4294985385 rc 301/301 00000100:00100000:0.0:1787826308.667055:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68386 00000100:00100000:3.0:1787826308.677595:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826308.677644:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332580992 00000100:00100000:3.0:1787826308.677655:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332580992 00000100:00100000:3.0:1787826308.677732:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9334 00000100:00100000:3.0:1787826308.677745:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224713:x1874663332580992:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:0.0:1787826308.685634:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd0450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224713:x1874663332580992:12345-192.168.203.56@tcp:37:rm.0 Request processed in 7881us (8049us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.685647:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9334 00000100:00100000:1.0:1787826308.708682:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826308.708761:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332581120 00000100:00100000:2.0:1787826308.708773:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332581120 00000100:00100000:2.0:1787826308.708826:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9335 00000100:00100000:2.0:1787826308.708836:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224713:x1874663332581120:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826308.719478:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224713:x1874663332581120:12345-192.168.203.56@tcp:37:rm.0 Request processed in 10630us (10804us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.719496:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9335 00000100:00100000:1.0:1787826308.736860:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826308.738037:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332581376 00000100:00100000:0.0:1787826308.738049:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332581376 00000100:00100000:0.0:1787826308.738285:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9336 00000100:00100000:0.0:1787826308.738297:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224713:x1874663332581376:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:0.0:1787826308.738554:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1196, 344 off1088608 00000040:00080000:0.0:1787826308.738603:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1514, 424 off1529752 00000004:00080000:0.0:1787826308.738609:0:6530:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784f7d7540 with version 6821 00000004:00080000:2.0:1787826308.740046:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6821 rpc_version 6821 00000020:00100000:2.0:1787826308.740086:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826308.740095:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9fd880 x1874663346192768/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826308.740134:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fd880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346192768:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826308.740198:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826308.740234:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826308.740259:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346192768 00000100:00100000:2.0:1787826308.740265:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346192768 00000100:00100000:2.0:1787826308.740309:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23591 00000100:00100000:2.0:1787826308.740318:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346192768:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826308.740506:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346192768:12345-0@lo:1000:osp_up0-1.0 Request processed in 188us (307us total) trans 8589957374 rc 0/0 00000100:00100000:2.0:1787826308.740514:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23591 00000100:00100000:2.0:1787826308.740570:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fd880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346192768:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826308.741570:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224713:x1874663332581376:12345-192.168.203.56@tcp:35:rm.0 Request processed in 3272us (4718us total) trans 4294985386 rc 0/0 00000100:00100000:0.0:1787826308.741580:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9336 00000100:00100000:1.0:1787826308.755424:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826308.755554:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332581632 00000100:00100000:3.0:1787826308.755564:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332581632 00000100:00100000:3.0:1787826308.755628:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68387 00000100:00100000:3.0:1787826308.755639:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224713:x1874663332581632:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826308.755662:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826308.755698:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a736 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013ae expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.755859:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a73d lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.755884:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a73d lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.755896:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a73d lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826308.755910:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4a:0x0]: returning size 4096 00010000:00010000:3.0:1787826308.756008:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a736 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013ae expref: 16 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.756029:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a73d lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013ae expref: 15 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.756038:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:3.0:1787826308.756041:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1787826308.756044:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826308.756133:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:224713:x1874663332581632:12345-192.168.203.56@tcp:101:rm.0 Request processed in 496us (718us total) trans 4294985387 rc 0/0 00000100:00100000:3.0:1787826308.756140:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68387 00000100:00100000:1.0:1787826308.777087:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826308.777153:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332581760 00000100:00100000:1.0:1787826308.777163:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332581760 00000100:00100000:1.0:1787826308.777503:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68388 00000100:00100000:1.0:1787826308.777519:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224713:x1874663332581760:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:1.0:1787826308.777572:0:6527:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808684032 avail 7345610752 00000100:00100000:1.0:1787826308.777644:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224713:x1874663332581760:12345-192.168.203.56@tcp:41:rm.0 Request processed in 129us (566us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826308.777653:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68388 00000100:00100000:0.0:1787826308.790042:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826308.790140:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332581888 00000100:00100000:1.0:1787826308.790151:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332581888 00000100:00100000:1.0:1787826308.790209:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68389 00000100:00100000:1.0:1787826308.790233:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:224713:x1874663332581888:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826308.790259:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826308.790281:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a74b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013b5 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790346:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790359:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790370:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790381:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790388:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790393:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790610:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a760 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790620:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a760 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790628:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a760 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826308.790648:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a760 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790675:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790683:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790689:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790695:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790701:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826308.790706:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790712:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790724:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe300) 00010000:00010000:1.0:1787826308.790727:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826308.790732:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790739:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a752 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790752:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790758:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790764:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790770:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826308.790772:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790778:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790785:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f784b4a5900) 00010000:00010000:1.0:1787826308.790788:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826308.790791:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790796:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4a759 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790824:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a74b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013b5 expref: 17 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790840:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a760 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013b5 expref: 16 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.790847:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:1.0:1787826308.790850:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826308.790852:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826308.790913:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:224713:x1874663332581888:12345-192.168.203.56@tcp:101:rm.0 Request processed in 693us (881us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826308.790925:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68389 00000100:00100000:1.0:1787826308.807575:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826308.809333:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332582016 00000100:00100000:0.0:1787826308.809348:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332582016 00000100:00100000:0.0:1787826308.809627:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68390 00000100:00100000:0.0:1787826308.809640:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224713:x1874663332582016:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826308.809662:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826308.809694:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a767 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013bc expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.809755:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a76e lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.809766:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a76e lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.809776:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a76e lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826308.809812:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1db9:0x0]: returning size 6 00000004:00200000:0.0:1787826308.809816:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:0.0:1787826308.809850:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a767 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013bc expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.809867:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a76e lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013bc expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.809876:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f79460a2100) 00010000:00010000:0.0:1787826308.809879:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826308.809883:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826308.810126:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224713:x1874663332582016:12345-192.168.203.56@tcp:101:rm.0 Request processed in 466us (2535us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.810135:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68390 00000100:00100000:2.0:1787826308.814759:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826308.814847:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332582144 00000100:00100000:3.0:1787826308.814858:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332582144 00000100:00100000:3.0:1787826308.814945:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11071 00000100:00100000:3.0:1787826308.814958:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:2393:x1874663332582144:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826308.814972:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826308.815002:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a760 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013b5 expref: 16 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.815020:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:3.0:1787826308.815029:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826308.815083:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:2393:x1874663332582144:12345-192.168.203.56@tcp:103:rm.0 Request processed in 128us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.815090:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11071 00000100:00100000:0.0:1787826308.817208:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826308.817245:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332582400 00000100:00100000:1.0:1787826308.817256:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332582400 00000100:00100000:1.0:1787826308.818473:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68391 00000100:00100000:1.0:1787826308.818493:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224713:x1874663332582400:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826308.818551:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826308.818582:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4a76e lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1db9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013bc expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.818603:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f79460a2100) 00010000:00010000:1.0:1787826308.818608:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826308.818701:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346192896:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787826308.818766:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826308.818808:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f797cd23f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826308.819733:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346192896 00000100:00100000:2.0:1787826308.819744:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346192896 00000100:00100000:2.0:1787826308.819804:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23592 00000100:00100000:2.0:1787826308.819813:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6528:x1874663346192896:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826308.819917:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6528:x1874663346192896:12345-0@lo:1000:mdt00_001.0 Request processed in 104us (1149us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.819924:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23592 00000100:00100000:2.0:1787826308.819972:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346192896:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787826308.820049:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.820059:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.820069:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826308.820085:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346193024:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787826308.820134:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826308.820163:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826308.820275:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346193024 00000100:00100000:1.0:1787826308.820281:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346193024 00000100:00100000:1.0:1787826308.820380:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23593 00000100:00100000:1.0:1787826308.820389:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6528:x1874663346193024:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787826308.820400:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826308.820417:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a77c lrc: 2/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a775 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.820444:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.820459:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a77c lrc: 4/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a775 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.820485:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.820498:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 14 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.820508:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346193152:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826308.820579:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f794620db40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826308.832852:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826308.832932:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332582528 00000100:00100000:3.0:1787826308.832943:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332582528 00010000:00010000:3.0:1787826308.832993:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 15 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.833037:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21925 00000100:00100000:3.0:1787826308.833049:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:2394:x1874663332582528:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826308.833065:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826308.833073:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 15 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.833088:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 15 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.833132:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe300) 00010000:00010000:3.0:1787826308.833142:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a77c lrc: 3/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a775 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.833154:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a77c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a775 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.833164:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a77c lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a775 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.833200:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a77c lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a775 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.833214:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:8459:1874663346193280:0@lo:105: 00000100:00100000:3.0:1787826308.833265:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.833315:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826308.834817:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346193280 00000100:00100000:2.0:1787826308.834832:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346193280 00000100:00100000:2.0:1787826308.834925:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7851 00000100:00100000:2.0:1787826308.834937:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:8459:x1874663346193280:12345-0@lo:105: 00010000:00010000:2.0:1787826308.834956:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.834968:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.834977:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.834986:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.834995:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.835005:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.835023:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef200) 00000100:00100000:2.0:1787826308.835143:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:8459:x1874663346193280:12345-0@lo:105: Request processed in 208us (1876us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.835152:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7851 00000100:00100000:3.0:1787826308.836027:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:8459:1874663346193280:0@lo:105: 00010000:00010000:3.0:1787826308.836054:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826308.836136:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:2394:x1874663332582528:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3086us (3292us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.836146:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21925 00010000:00010000:1.0:1787826308.838186:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a713 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801399 expref: 14 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.838215:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346193152:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826308.838251:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a77c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a775 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.838263:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe300) 00010000:00010000:1.0:1787826308.838266:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826308.838270:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826308.838429:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346193024:12345-0@lo:101:mdt00_001.0 Request processed in 18040us (18293us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826308.838438:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23593 00000100:00100000:2.0:1787826308.838513:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346193024:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826308.838537:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826308.838576:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x193:0x0] 00010000:00010000:2.0:1787826308.838633:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.838642:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.838654:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826308.838684:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346193408:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787826308.838724:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826308.838760:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.838771:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346193408 00000100:00100000:3.0:1787826308.838780:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346193408 00000100:00100000:3.0:1787826308.838830:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23594 00000100:00100000:3.0:1787826308.838839:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346193408:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787826308.838854:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826308.838870:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a783 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.838902:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 2/0,1 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.838912:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a783 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.838928:0:6533:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:00080000:3.0:1787826308.838961:0:6533:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00010000:00010000:3.0:1787826308.839007:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a783 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.839021:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f784b4a5900) 00010000:00010000:3.0:1787826308.839023:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826308.839027:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a783 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.839035:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826308.839080:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6528:x1874663346193408:12345-0@lo:101:mdt00_001.0 Request processed in 242us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.839087:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23594 00000100:00100000:0.0:1787826308.839243:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346193408:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787826308.839256:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.839264:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.852712:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826308.852734:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.852779:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.852786:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826308.852838:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346193536:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826308.852919:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826308.855688:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4a6b8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6bf expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.855727:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a6d4 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826308.855736:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a6d4 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.855781:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6d4 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826308.855784:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346193536 00010000:00010000:3.0:1787826308.855786:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826308.855794:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346193536 00010000:00010000:3.0:1787826308.855811:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4a6cd lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a6d4 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826308.855826:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985387 is committed 00000100:00100000:2.0:1787826308.855851:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11072 00000100:00100000:2.0:1787826308.855862:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd81050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346193536:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826308.855876:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826308.855907:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a6bf lrc: 0/0,0 mode: --/PW res: [0x200000401:0x191:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a6b8 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.855924:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x4ec94fd6 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826308.855935:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000001:00080000:3.0:1787826308.855971:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985384 is committed 00000004:00080000:3.0:1787826308.855978:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 364 committed 00000001:00080000:3.0:1787826308.855994:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985382 is committed 00000020:00080000:3.0:1787826308.856012:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3409 batchid updated 00000004:00080000:3.0:1787826308.856021:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 402 committed 00000100:00100000:1.0:1787826308.856029:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346193664:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787826308.856046:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856056:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856063:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856070:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00000100:00100000:1.0:1787826308.856076:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826308.856076:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856086:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856098:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f784b4a5900) 00010000:00010000:3.0:1787826308.856102:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826308.856136:0:3638:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a783 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856148:0:3638:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a783 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856157:0:3638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a783 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.856158:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346193664 00000100:00100000:1.0:1787826308.856163:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346193664 00000100:00100000:1.0:1787826308.856198:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11073 00010000:00010000:3.0:1787826308.856208:0:3638:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a783 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.856209:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346193664:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826308.856223:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826308.856229:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0001_UUID:3638:1874663346193792:0@lo:105: 00010000:00010000:1.0:1787826308.856239:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4a6d4 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a6cd expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.856253:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f79460a2100) 00010000:00010000:1.0:1787826308.856261:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826308.856265:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.856300:0:3638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826308.856313:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346193664:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 106us (237us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826308.856321:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11073 00000100:00100000:1.0:1787826308.856370:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346193664:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:3.0:1787826308.856387:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3409 00000100:00100000:1.0:1787826308.856389:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826308.856397:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346193920:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826308.856428:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1787826308.856445:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7850703cc0 with version 6822 00000020:00080000:3.0:1787826308.856456:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3406 cancel update log [0x200000401:0x3:0x0].1509: rc = 0 00000020:00080000:3.0:1787826308.856646:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3406 cancel update log [0x240000400:0x3:0x0].1191: rc = 0 00000004:00080000:3.0:1787826308.856686:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7850703d80 with version 6823 00000020:00080000:3.0:1787826308.856690:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3407 cancel update log [0x200000401:0x3:0x0].1510: rc = 0 00000020:00080000:3.0:1787826308.856713:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3407 cancel update log [0x240000400:0x3:0x0].1192: rc = 0 00000004:00080000:3.0:1787826308.856731:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78453cec00 with version 6824 00000020:00080000:3.0:1787826308.856734:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3408 cancel update log [0x200000401:0x3:0x0].1511: rc = 0 00000020:00080000:3.0:1787826308.856744:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3408 cancel update log [0x240000400:0x3:0x0].1193: rc = 0 00000004:00080000:3.0:1787826308.856755:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78453ce540 with version 6825 00000020:00080000:3.0:1787826308.856757:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3409 cancel update log [0x200000401:0x3:0x0].1512: rc = 0 00000020:00080000:3.0:1787826308.856765:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3409 cancel update log [0x240000400:0x3:0x0].1194: rc = 0 00000020:00080000:3.0:1787826308.856770:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3409 00000100:00100000:3.0:1787826308.856784:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346193792 00000100:00100000:3.0:1787826308.856788:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346193792 00000100:00100000:3.0:1787826308.856818:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7852 00000100:00100000:3.0:1787826308.856823:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:3638:x1874663346193792:12345-0@lo:105: 00010000:00010000:3.0:1787826308.856833:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856848:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856857:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856871:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 5/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856878:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 6/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.856886:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 6/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.857166:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef600) 00000100:00100000:3.0:1787826308.857207:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:3638:x1874663346193792:12345-0@lo:105: Request processed in 383us (945us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.857215:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7852 00000100:00100000:3.0:1787826308.857238:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346193920 00000100:00100000:3.0:1787826308.857242:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346193920 00000100:00100000:3.0:1787826308.857324:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30818 00000100:00100000:3.0:1787826308.857332:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346193920:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:3.0:1787826308.857354:0:20167:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1228 count 1 00010000:00010000:3.0:1787826308.857376:0:20167:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.857395:0:20167:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.857403:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=2s v=(1 0) 00010000:00010000:3.0:1787826308.857408:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=0s v=(0 1) 00010000:00010000:3.0:1787826308.857413:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.857437:0:20167:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.857450:0:20167:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 5 pid: 35352 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.857462:0:20167:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346194048:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826308.858089:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.858124:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826308.858153:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x193:0x0] 00000100:00100000:3.0:1787826308.858212:0:20167:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f540 going to sleep for 11 seconds 00000004:00080000:3.0:1787826308.858230:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6822 rpc_version 6822 00000020:00100000:3.0:1787826308.858251:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826308.858259:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977e0f480 x1874663346194432/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826308.858274:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0f480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346194432:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826308.858315:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.858337:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00080000:3.0:1787826308.858350:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826308.858368:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346194432 00000100:00100000:3.0:1787826308.858373:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346194432 00000100:00100000:3.0:1787826308.858410:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23595 00000100:00100000:3.0:1787826308.858419:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346194432:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826308.858463:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346194176:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787826308.858500:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.858561:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826308.858630:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd81050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346193536:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 2769us (5709us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.858638:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11072 00000100:00100000:3.0:1787826308.860230:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346194432:12345-0@lo:1000:osp_up0-1.0 Request processed in 1804us (1906us total) trans 8589957375 rc 0/0 00000100:00100000:3.0:1787826308.860244:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23595 00000100:00100000:3.0:1787826308.860276:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346194176 00000100:00100000:3.0:1787826308.860283:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346194176 00000100:00100000:2.0:1787826308.860407:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0f480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346194432:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826308.860422:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6823 rpc_version 6823 00000020:00100000:2.0:1787826308.860445:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826308.860452:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784837aa00 x1874663346194688/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826308.860472:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346194688:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826308.860519:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.864891:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346193536:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826308.865094:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826308.865141:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332582656 00000100:00100000:0.0:1787826308.865148:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332582656 00010000:00010000:0.0:1787826308.865190:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 6 pid: 35352 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.865238:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21926 00000100:00100000:0.0:1787826308.865249:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332582656:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826308.865264:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826308.865270:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 6 pid: 35352 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865285:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 6 pid: 35352 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865324:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=0s v=(0 1) 00010000:00010000:0.0:1787826308.865329:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cc:0x0].0x0 since=0s v=(0 1) 00010000:00010000:0.0:1787826308.865335:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865346:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865368:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826308.865424:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332582656:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 175us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.865432:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21926 00010000:00010000:0.0:1787826308.865558:0:20167:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008638212c/0xe21309aa81f4a6c6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801384 expref: 5 pid: 35352 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.865579:0:20167:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346194048:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826308.865598:0:20167:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865609:0:20167:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865617:0:20167:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865627:0:20167:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865635:0:20167:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865652:0:20167:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826308.865855:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346193920:12345-0@lo:6:osp-syn-0-1.0 Request processed in 8523us (9425us total) trans 4294973846 rc 0/0 00000100:00100000:0.0:1787826308.865864:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30818 00010000:00010000:0.0:1787826308.865891:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826308.865898:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865910:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826308.865915:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865924:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865937:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865944:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acfa8029/0xe21309aa81f4a791 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.865961:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826308.865962:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826308.865964:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1787826308.865988:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7977e0c700/2, rc 0, transno 4294973846 00000100:00100000:0.0:1787826308.865994:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346193920:0@lo:6:osp-syn-0-1.0 00000100:00100000:2.0:1787826308.866227:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.866452:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346194688 00000100:00100000:3.0:1787826308.866461:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346194688 00000100:00100000:3.0:1787826308.866494:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23596 00000100:00100000:3.0:1787826308.866504:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346194176:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787826308.866596:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346194176:12345-0@lo:1000:mdt00_001.0 Request processed in 94us (8095us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.866603:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23596 00000100:00100000:3.0:1787826308.866618:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23597 00000100:00100000:3.0:1787826308.866624:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346194688:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826308.866741:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346194688:12345-0@lo:1000:osp_up0-1.0 Request processed in 116us (6220us total) trans 8589957376 rc 0/0 00000100:00100000:3.0:1787826308.866750:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23597 00000100:00100000:3.0:1787826308.866788:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346194688:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826308.866799:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6824 rpc_version 6824 00000020:00100000:3.0:1787826308.866819:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826308.866826:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977e0df80 x1874663346194944/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826308.866845:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0df80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346194944:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826308.866878:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.866894:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.866917:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0001_UUID:3638:1874663346193792:0@lo:105: 00010000:00010000:3.0:1787826308.866936:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.866945:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4a69c lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.866973:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346194944 00000100:00100000:3.0:1787826308.866976:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346194944 00000100:00100000:3.0:1787826308.866995:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23598 00000100:00100000:3.0:1787826308.867001:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346194944:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826308.867074:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346194944:12345-0@lo:1000:osp_up0-1.0 Request processed in 72us (195us total) trans 8589957377 rc 0/0 00000100:00100000:3.0:1787826308.867081:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23598 00000100:00100000:3.0:1787826308.867133:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0df80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346194944:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826308.867140:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6825 rpc_version 6825 00000020:00100000:3.0:1787826308.867151:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826308.867156:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977e0d180 x1874663346195200/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826308.867168:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0d180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346195200:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826308.867194:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.867209:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.867220:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346195200 00000100:00100000:3.0:1787826308.867223:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346195200 00000100:00100000:3.0:1787826308.867240:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23599 00000100:00100000:3.0:1787826308.867247:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346195200:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826308.867313:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346195200:12345-0@lo:1000:osp_up0-1.0 Request processed in 66us (116us total) trans 8589957378 rc 0/0 00000100:00100000:3.0:1787826308.867319:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23599 00000100:00100000:3.0:1787826308.867347:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0d180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346195200:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826308.867520:0:6528:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985382 gen 2 00000004:00080000:2.0:1787826308.867526:0:6528:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985382 committed transno 4294985382 00000100:00100000:2.0:1787826308.867542:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346194176:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826308.867594:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346195328:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826308.867621:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826308.867641:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.867687:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346195328 00000100:00100000:3.0:1787826308.867692:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346195328 00000100:00100000:3.0:1787826308.867723:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23600 00000100:00100000:3.0:1787826308.867731:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346195328:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787826308.867792:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346195328:12345-0@lo:1000:mdt00_001.0 Request processed in 62us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.867799:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23600 00000100:00100000:3.0:1787826308.867824:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346195328:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787826308.867864:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346195456:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787826308.867888:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.867902:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.867912:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346195456 00000100:00100000:3.0:1787826308.867915:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346195456 00000100:00100000:3.0:1787826308.867934:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23601 00000100:00100000:3.0:1787826308.867940:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346195456:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787826308.868005:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346195456:12345-0@lo:1000:mdt00_001.0 Request processed in 64us (115us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826308.868011:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23601 00000100:00100000:3.0:1787826308.868037:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346195456:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1787826308.868089:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868099:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868136:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.868147:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346195584:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787826308.868169:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.868183:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.868194:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346195584 00000100:00100000:3.0:1787826308.868198:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346195584 00000100:00100000:3.0:1787826308.868215:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23602 00000100:00100000:3.0:1787826308.868221:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6528:x1874663346195584:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787826308.868229:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826308.868243:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4a79f lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a798 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868266:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4a79f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a798 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868284:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4a79f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a798 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868292:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:3.0:1787826308.868295:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826308.868298:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826308.868326:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6528:x1874663346195584:12345-0@lo:101:mdt00_001.0 Request processed in 105us (156us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.868332:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23602 00000100:00100000:3.0:1787826308.868356:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6528:1874663346195584:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787826308.868367:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868377:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868399:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868408:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.868416:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787826308.868750:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].212, 64 off21760 00000004:00080000:3.0:1787826308.868763:0:6528:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8121080657300ns, next at 8122080654690ns, rc = 0 00000040:00080000:3.0:1787826308.868795:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].934, 384 off878248 00000040:00080000:3.0:1787826308.868834:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].937, 464 off977312 00000004:00080000:3.0:1787826308.868839:0:6528:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784549fd80 with version 1823 00000004:00080000:1.0:1787826308.869052:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1823 rpc_version 1823 00000020:00100000:1.0:1787826308.869074:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826308.869082:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7849dd0a80 x1874663346195840/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1664/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826308.869097:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346195840:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826308.869298:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826308.869399:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f794620db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826308.869435:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346195840 00000100:00100000:2.0:1787826308.869442:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346195840 00000100:00100000:2.0:1787826308.869504:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23603 00000100:00100000:2.0:1787826308.869515:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346195840:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826308.869831:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346195840:12345-0@lo:1000:osp_up1-0.0 Request processed in 317us (532us total) trans 4294985388 rc 0/0 00000100:00100000:2.0:1787826308.869839:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23603 00000100:00100000:1.0:1787826308.869918:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346195840:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787826308.869981:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.869994:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870003:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870014:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870020:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870026:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870078:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826308.870098:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870130:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870137:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870142:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826308.870147:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870154:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870164:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f79460a2700) 00010000:00010000:0.0:1787826308.870167:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826308.870171:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870181:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4a7a6 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870193:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870200:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870207:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870402:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826308.870409:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870415:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826308.870419:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870426:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870435:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f79460a2700) 00010000:00010000:1.0:1787826308.870438:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826308.870440:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870446:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4a7ad lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1db9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870461:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826308.870465:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870471:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826308.870475:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 5/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826308.870482:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 5/0,0 mode: EX/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870508:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 4/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870515:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826308.870570:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:1.0:1787826308.870614:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 3/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870624:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826308.870626:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826308.870628:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826308.870688:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1787826308.870693:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4a798 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a79f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826308.870718:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x193:0x0] 00010000:00010000:0.0:1787826308.870725:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870732:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.870739:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.870771:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851886680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346195968:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826308.870815:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826308.870893:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346195968 00000100:00100000:1.0:1787826308.870899:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346195968 00000100:00100000:1.0:1787826308.870950:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11074 00000100:00100000:1.0:1787826308.870959:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346195968:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826308.870970:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826308.870985:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4a79f lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1db9:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a798 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.870996:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1db9:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:1.0:1787826308.871003:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826308.871040:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346195968:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826308.871047:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11074 00000100:00100000:1.0:1787826308.871080:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851886680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346195968:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826308.871266:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826308.871272:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.871280:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826308.871284:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826308.871291:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.871314:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 4/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.871320:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826308.871338:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.871347:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826308.871348:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826308.871350:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826308.871383:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346196096:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826308.871409:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826308.871470:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346196096 00000100:00100000:2.0:1787826308.871475:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346196096 00000100:00100000:2.0:1787826308.871510:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11075 00000100:00100000:2.0:1787826308.871517:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346196096:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826308.871526:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826308.871537:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a77c lrc: 0/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a775 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.871549:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784abfe300) 00010000:00010000:2.0:1787826308.871557:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826308.871587:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346196096:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.871593:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11075 00000100:00100000:1.0:1787826308.871651:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346196096:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826308.871784:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1787826308.871791:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef200/0xe21309aa81f4a775 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a77c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826308.871812:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x193:0x0] 00010000:00010000:0.0:1787826308.871818:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.871825:0:6528:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.871903:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224713:x1874663332582400:12345-192.168.203.56@tcp:36:rm.0 Request processed in 53414us (54703us total) trans 8589957379 rc 0/0 00000100:00100000:0.0:1787826308.871910:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68391 00010000:00010000:2.0:1787826308.872073:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826308.877775:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826308.877827:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826308.877902:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332582784 00000100:00100000:2.0:1787826308.877913:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332582784 00000100:00100000:2.0:1787826308.877982:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11076 00000100:00100000:2.0:1787826308.877993:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332582784:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826308.878008:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826308.878031:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a5d1 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801337 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.878050:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:2.0:1787826308.878055:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826308.878136:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332582784:12345-192.168.203.56@tcp:103:rm.0 Request processed in 145us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.878145:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11076 00000100:00100000:1.0:1787826308.880291:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332583040 00000100:00100000:1.0:1787826308.880308:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332583040 00000100:00100000:0.0:1787826308.882340:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9337 00000100:00100000:0.0:1787826308.882357:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224713:x1874663332583040:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:0.0:1787826308.882591:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224713:x1874663332583040:12345-192.168.203.56@tcp:35:rm.0 Request processed in 236us (4762us total) trans 4294985389 rc 0/0 00000100:00100000:0.0:1787826308.882599:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9337 00000100:00100000:3.0:1787826308.891999:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826308.892149:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332583296 00000100:00100000:0.0:1787826308.892160:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332583296 00000100:00100000:0.0:1787826308.892226:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68392 00000100:00100000:0.0:1787826308.892238:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224713:x1874663332583296:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826308.892298:0:35092:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826308.892326:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a73d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013ae expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.892355:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a705 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801392 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.892367:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a5df lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080133e expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.892377:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:0.0:1787826308.892382:0:35092:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826308.892428:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.892437:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.892448:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.892471:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346196224:0@lo:101:mdt00_009.0 00000100:00100000:0.0:1787826308.892523:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.892561:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826308.892622:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346196224 00000100:00100000:1.0:1787826308.892627:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346196224 00000100:00100000:1.0:1787826308.892687:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23604 00000100:00100000:1.0:1787826308.892695:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:35092:x1874663346196224:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826308.892707:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826308.892719:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a7b4 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.892740:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.892748:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.892756:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a7b4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.892780:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.892792:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 7 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.892803:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851884a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346196352:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826308.892867:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.892882:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 8 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.892892:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346196480:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826308.892918:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a8bfa80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826308.896383:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826308.896426:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826308.896507:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332583552 00000100:00100000:3.0:1787826308.896517:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332583552 00010000:00010000:3.0:1787826308.896559:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 9 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.896597:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332583424 00000100:00100000:3.0:1787826308.896600:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332583424 00010000:00010000:3.0:1787826308.896612:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 8 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.896631:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21927 00000100:00100000:3.0:1787826308.896640:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332583552:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826308.896654:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826308.896661:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 9 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.896689:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 9 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.896711:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826308.896718:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a7b4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.896733:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826308.896784:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332583552:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 144us (411us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.896791:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21927 00000100:00100000:3.0:1787826308.896809:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21928 00000100:00100000:3.0:1787826308.896815:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd80850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332583424:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826308.896822:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826308.896826:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 8 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.896836:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 8 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.896846:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826308.896851:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a7b4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.896859:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a7b4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.896866:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a7b4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.896898:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a7b4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.896917:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346196608:0@lo:105: 00000100:00100000:3.0:1787826308.896974:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.897009:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.897035:0:6509:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826308.897046:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346196608 00000100:00100000:3.0:1787826308.897049:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346196608 00000100:00100000:3.0:1787826308.897076:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7853 00000100:00100000:3.0:1787826308.897082:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843684050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346196608:12345-0@lo:105: 00010000:00010000:3.0:1787826308.897093:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.897102:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.897141:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.897150:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.897157:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.897166:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.897184:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42000) 00000100:00100000:3.0:1787826308.897215:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843684050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346196608:12345-0@lo:105: Request processed in 133us (241us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.897221:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7853 00000100:00100000:3.0:1787826308.897281:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346196608:0@lo:105: 00010000:00010000:3.0:1787826308.897295:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826308.897316:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd80850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332583424:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 501us (890us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.897322:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21928 00010000:00010000:0.0:1787826308.897807:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a61e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080135a expref: 7 pid: 6528 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.897827:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851884a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346196352:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826308.897840:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4a5b5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801330 expref: 8 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.897855:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346196480:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826308.897881:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a7b4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.897889:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826308.897892:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826308.897895:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826308.898024:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346196224:0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826308.898040:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826308.898060:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826308.898096:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.898133:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.898142:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.898159:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346196736:0@lo:101:mdt00_009.0 00000100:00100000:1.0:1787826308.898192:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826308.898216:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a8bf840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826308.898276:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:35092:x1874663346196224:12345-0@lo:101:mdt00_009.0 Request processed in 5580us (5749us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.898283:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23604 00000100:00100000:0.0:1787826308.898307:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346196736 00000100:00100000:0.0:1787826308.898311:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346196736 00000100:00100000:0.0:1787826308.898340:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23605 00000100:00100000:0.0:1787826308.898347:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:35092:x1874663346196736:12345-0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826308.898360:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826308.898374:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a7c9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a7c2 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.898394:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a7c9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a7c2 expref: 15 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.898405:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a7c9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a7c2 expref: 15 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.898413:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5700) 00010000:00010000:0.0:1787826308.898417:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826308.898419:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826308.898450:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:35092:x1874663346196736:12345-0@lo:101:mdt00_009.0 Request processed in 103us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.898455:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23605 00000100:00100000:0.0:1787826308.898480:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346196736:0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826308.898491:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.898501:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826308.898515:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826308.898547:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346196864:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826308.898573:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.898587:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826308.898603:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346196864 00000100:00100000:0.0:1787826308.898606:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346196864 00000100:00100000:0.0:1787826308.898628:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23606 00000100:00100000:0.0:1787826308.898634:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:35092:x1874663346196864:12345-0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826308.898702:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:35092:x1874663346196864:12345-0@lo:1000:mdt00_009.0 Request processed in 67us (128us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.898709:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23606 00000100:00100000:0.0:1787826308.898734:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346196864:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826308.898773:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346196992:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826308.898796:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.898809:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826308.898820:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346196992 00000100:00100000:0.0:1787826308.898823:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346196992 00000100:00100000:0.0:1787826308.898841:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23607 00000100:00100000:0.0:1787826308.898847:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:35092:x1874663346196992:12345-0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826308.899316:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:35092:x1874663346196992:12345-0@lo:1000:mdt00_009.0 Request processed in 466us (516us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826308.899324:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23607 00000100:00100000:0.0:1787826308.899355:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346196992:0@lo:1000:mdt00_009.0 00010000:00010000:0.0:1787826308.899408:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.899416:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.899423:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.899436:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346197120:0@lo:101:mdt00_009.0 00000100:00100000:0.0:1787826308.899462:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.899523:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826308.902763:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346197120 00000100:00100000:1.0:1787826308.902778:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346197120 00000100:00100000:1.0:1787826308.902850:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23608 00000100:00100000:1.0:1787826308.902862:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:35092:x1874663346197120:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826308.902881:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826308.902904:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a7d7 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a7d0 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.903070:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.903083:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a7d7 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a7d0 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.903305:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.903319:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 6 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.903333:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346197248:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826308.903456:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a8bfb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826308.912367:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826308.912456:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332583680 00000100:00100000:3.0:1787826308.912466:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332583680 00010000:00010000:3.0:1787826308.912508:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 5/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 7 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.912553:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21929 00000100:00100000:3.0:1787826308.912564:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332583680:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826308.912578:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826308.912585:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 5/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 7 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.912599:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 7 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.912622:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:3.0:1787826308.912630:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a7d7 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a7d0 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.912642:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a7d7 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a7d0 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.912651:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a7d7 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a7d0 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.912697:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a7d7 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a7d0 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826308.912713:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346197376:0@lo:105: 00000100:00100000:3.0:1787826308.912762:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.912846:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fa80 going to sleep for 11 seconds 00010000:00010000:2.0:1787826308.912936:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a63a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801361 expref: 7 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826308.912954:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346197248:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826308.912983:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a7d7 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a7d0 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826308.912993:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:2.0:1787826308.912996:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826308.912999:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:2.0:1787826308.913048:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+16:35092:x1874663346197120:12345-0@lo:101:mdt00_009.0 Request processed in 10189us (13585us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.913056:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23608 00000100:00100000:0.0:1787826308.913158:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346197120:0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826308.913175:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913186:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.913486:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346197376 00000100:00100000:1.0:1787826308.913491:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346197376 00000100:00100000:1.0:1787826308.913566:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7854 00000100:00100000:1.0:1787826308.913575:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346197376:12345-0@lo:105: 00010000:00010000:1.0:1787826308.913585:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.913600:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.913609:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.913618:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.913626:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.913634:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.913657:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42a00) 00000100:00100000:1.0:1787826308.913714:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346197376:12345-0@lo:105: Request processed in 141us (951us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826308.913720:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7854 00010000:00010000:0.0:1787826308.913761:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913771:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913805:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913816:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913825:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913848:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913856:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913863:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913888:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 3/0,1 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913895:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 3/0,1 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.913904:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 3/0,1 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.913919:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346197504:0@lo:101:mdt00_009.0 00000100:00100000:0.0:1787826308.913951:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.913972:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826308.914025:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346197376:0@lo:105: 00010000:00010000:2.0:1787826308.914037:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826308.914085:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332583680:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1522us (1726us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826308.914093:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21929 00000100:00100000:0.0:1787826308.915178:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346197504 00000100:00100000:0.0:1787826308.915200:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346197504 00000100:00100000:0.0:1787826308.915251:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23609 00000100:00100000:0.0:1787826308.915260:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+16:35092:x1874663346197504:12345-0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826308.915279:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826308.915298:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a7f3 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a7ec expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.915322:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.915331:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a7f3 lrc: 4/0,0 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a7ec expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.915359:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.915373:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 7 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.915384:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346197632:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826308.915459:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.918710:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826308.918787:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332583808 00000100:00100000:1.0:1787826308.918796:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332583808 00010000:00010000:1.0:1787826308.918839:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 8 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.918883:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21930 00000100:00100000:1.0:1787826308.918893:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332583808:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826308.918911:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826308.918918:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 8 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.918938:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 8 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.918963:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:1.0:1787826308.918971:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a7f3 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a7ec expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.918983:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a7f3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a7ec expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.918994:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a7f3 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a7ec expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826308.919028:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a7f3 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a7ec expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.919043:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346197760:0@lo:105: 00000100:00100000:1.0:1787826308.919090:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826308.919153:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a8bfb40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.919719:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346197760 00000100:00100000:3.0:1787826308.919726:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346197760 00000100:00100000:3.0:1787826308.919771:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7855 00000100:00100000:3.0:1787826308.919780:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346197760:12345-0@lo:105: 00010000:00010000:3.0:1787826308.919792:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 4/0,1 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.919803:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 4/0,1 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.919812:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 4/0,1 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.919820:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.919828:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.919836:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826308.919851:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42400) 00000100:00100000:3.0:1787826308.919902:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346197760:12345-0@lo:105: Request processed in 123us (810us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.919909:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7855 00000100:00100000:1.0:1787826308.920066:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346197760:0@lo:105: 00010000:00010000:1.0:1787826308.920088:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826308.920159:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332583808:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1266us (1457us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826308.920172:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21930 00010000:00010000:0.0:1787826308.923096:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a721 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x191:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013a0 expref: 7 pid: 35092 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.923299:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346197632:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826308.923342:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a7f3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a7ec expref: 17 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.923354:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:0.0:1787826308.923357:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826308.923361:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:1.0:1787826308.923735:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+17:35092:x1874663346197504:12345-0@lo:101:mdt00_009.0 Request processed in 8474us (9781us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826308.923745:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23609 00000100:00100000:0.0:1787826308.923824:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346197504:0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826308.923842:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826308.925534:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321180 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346197888:0@lo:403:mdt00_009.0 00000100:00100000:1.0:1787826308.925611:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826308.925700:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a8bfb40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826308.925752:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346197888 00000100:00100000:3.0:1787826308.925760:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346197888 00000100:00100000:3.0:1787826308.925858:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23610 00000100:00100000:3.0:1787826308.925868:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+17:35092:x1874663346197888:12345-0@lo:403:mdt00_009.0 00000100:00100000:3.0:1787826308.931632:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+17:35092:x1874663346197888:12345-0@lo:403:mdt00_009.0 Request processed in 5756us (6011us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826308.931645:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23610 00000100:00100000:1.0:1787826308.931907:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321180 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346197888:0@lo:403:mdt00_009.0 00000040:00080000:1.0:1787826308.933390:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1197, 952 off1089560 00000040:00080000:1.0:1787826308.933553:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1515, 1032 off1530784 00000004:00080000:1.0:1787826308.933559:0:35092:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784a8bfb40 with version 6826 00000004:00080000:3.0:1787826308.933647:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6826 rpc_version 6826 00000020:00100000:3.0:1787826308.933688:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826308.933698:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977c6fb80 x1874663346198144/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826308.933722:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346198144:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826308.933781:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826308.933813:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826308.933873:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346198144 00000100:00100000:2.0:1787826308.933880:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346198144 00000100:00100000:2.0:1787826308.933957:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23611 00000100:00100000:2.0:1787826308.933966:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:7471:x1874663346198144:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826308.934308:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+18:7471:x1874663346198144:12345-0@lo:1000:osp_up0-1.0 Request processed in 342us (525us total) trans 8589957380 rc 0/0 00000100:00100000:2.0:1787826308.934315:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23611 00000100:00100000:3.0:1787826308.934401:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6fb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346198144:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787826308.934612:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4a:0x0]: returning size 4096 00010000:00010000:0.0:1787826308.934634:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934643:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934651:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934657:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826308.934662:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934679:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934700:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:0.0:1787826308.934703:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826308.934708:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934715:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4a7e5 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934737:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934746:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934753:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934782:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826308.934792:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x191:0x0] 00010000:00010000:0.0:1787826308.934801:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934806:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934813:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934819:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826308.934821:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934828:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934836:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:0.0:1787826308.934838:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826308.934841:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934846:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a7de lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934855:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934862:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934868:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934883:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787826308.934892:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826308.934897:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934904:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934911:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934925:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787826308.934928:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826308.934945:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.934952:0:35092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826308.935024:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224713:x1874663332583296:12345-192.168.203.56@tcp:36:rm.0 Request processed in 42787us (43032us total) trans 4294985390 rc 0/0 00000100:00100000:0.0:1787826308.935033:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68392 00010000:00010000:0.0:1787826308.935061:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787826308.935067:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935074:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826308.935078:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826308.935086:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935135:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 2/0,0 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935143:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826308.935163:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 1/0,0 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935171:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a7ec lrc: 0/0,0 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7f3 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935196:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826308.935200:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935206:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826308.935210:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826308.935216:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935232:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935237:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826308.935242:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935248:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42a00/0xe21309aa81f4a7d0 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7d7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935258:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826308.935261:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935271:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826308.935274:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826308.935281:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935295:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935301:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826308.935317:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935324:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a7b4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a7bb expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935339:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826308.935340:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826308.935344:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826308.935356:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346198272:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826308.935399:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.935428:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826308.935433:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346198400:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826308.935453:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.935479:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346198528:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826308.935502:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826308.935532:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346198272 00000100:00100000:0.0:1787826308.935537:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346198272 00000100:00100000:0.0:1787826308.935575:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346198400 00000100:00100000:0.0:1787826308.935578:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346198400 00000100:00100000:0.0:1787826308.935597:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346198528 00000100:00100000:0.0:1787826308.935600:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346198528 00000100:00100000:0.0:1787826308.935619:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11077 00000100:00100000:0.0:1787826308.935627:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+20:3654:x1874663346198272:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826308.935639:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826308.935655:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a7f3 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x191:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a7ec expref: 20 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935678:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x191:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:0.0:1787826308.935686:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826308.935715:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+19:3654:x1874663346198272:12345-0@lo:103:ldlm_bl.0 Request processed in 88us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.935722:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11077 00000100:00100000:0.0:1787826308.935737:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11078 00000100:00100000:0.0:1787826308.935743:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+18:3654:x1874663346198400:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826308.935750:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826308.935762:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a7d7 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a7d0 expref: 18 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935772:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:0.0:1787826308.935779:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826308.935799:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+17:3654:x1874663346198400:12345-0@lo:103:ldlm_bl.0 Request processed in 56us (344us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.935804:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11078 00000100:00100000:0.0:1787826308.935817:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11079 00000100:00100000:0.0:1787826308.935822:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+16:3652:x1874663346198528:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826308.935829:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826308.935840:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4a7bb lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a7b4 expref: 16 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826308.935849:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826308.935852:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826308.935876:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346198528:12345-0@lo:103:ldlm_bl.0 Request processed in 54us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826308.935882:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11079 00000100:00100000:0.0:1787826308.935924:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346198528:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826308.935949:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346198400:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826308.935957:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346198272:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826308.936269:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826308.939576:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826309.017621:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.017764:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332583936 00000100:00100000:0.0:1787826309.017775:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332583936 00000100:00100000:0.0:1787826309.017863:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68393 00000100:00100000:0.0:1787826309.017874:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224721:x1874663332583936:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:0.0:1787826309.017932:0:35094:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808684032 avail 7345610752 00000100:00100000:0.0:1787826309.018018:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224721:x1874663332583936:12345-192.168.203.56@tcp:41:stat.0 Request processed in 146us (408us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.018026:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68393 00000100:00080000:3.0:1787826309.018183:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826309.069571:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826309.069679:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332584064 00000100:00100000:3.0:1787826309.069691:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332584064 00000100:00100000:3.0:1787826309.069757:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68394 00000100:00100000:3.0:1787826309.069769:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224725:x1874663332584064:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826309.069791:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826309.069815:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a7fa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013c3 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.069885:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.069898:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.069908:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826309.069966:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826309.069971:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826309.069991:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.070015:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a7fa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.070034:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.070047:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826309.070049:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826309.070053:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826309.070159:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224725:x1874663332584064:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 392us (599us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.070166:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68394 00000100:00100000:2.0:1787826309.078468:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826309.080809:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332584192 00000100:00100000:1.0:1787826309.080826:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332584192 00000100:00100000:1.0:1787826309.080910:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68395 00000100:00100000:1.0:1787826309.080923:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224725:x1874663332584192:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826309.080953:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826309.080977:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a808 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013ca expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.081048:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.081062:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.081075:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.081093:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.081155:0:6527:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a7c9 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a7c2 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.081166:0:6527:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.081377:0:6527:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a7c9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a7c2 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826309.081395:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346198656:0@lo:104: 00000100:00100000:1.0:1787826309.081463:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826309.081515:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78555e7c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826309.081587:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346198656 00000100:00100000:2.0:1787826309.081593:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346198656 00000100:00100000:2.0:1787826309.081638:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7856 00000100:00100000:2.0:1787826309.081647:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346198656:12345-0@lo:104: 00010000:00010000:2.0:1787826309.081660:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.081733:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826309.081739:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346198656:12345-0@lo:104: Request processed in 94us (277us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826309.081747:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7856 00010000:00010000:2.0:1787826309.081771:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826309.081776:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.081784:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826309.081789:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826309.081798:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.081823:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.081830:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826309.081848:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.081855:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfa00/0xe21309aa81f4a7c2 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a7c9 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.081874:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826309.081877:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826309.081879:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826309.082052:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346198656:0@lo:104: 00010000:00010000:1.0:1787826309.082077:0:6527:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.082613:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826309.082628:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346198784:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826309.082685:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.082849:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346198784 00000100:00100000:2.0:1787826309.082856:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346198784 00010000:00010000:2.0:1787826309.082880:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a7c9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a7c2 expref: 15 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.082912:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21931 00000100:00100000:2.0:1787826309.082920:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346198784:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826309.082943:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826309.092418:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826309.092447:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.092507:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.092516:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826309.092611:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346198912:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826309.092720:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.092830:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346198912 00000100:00100000:2.0:1787826309.092841:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346198912 00000100:00100000:2.0:1787826309.092914:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11080 00000100:00100000:2.0:1787826309.092924:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd82850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346198912:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826309.092939:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826309.092961:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a78a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a783 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.092982:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x193:0x0].0x5a734ce9 (ffff9f784b4a5900) 00010000:00010000:2.0:1787826309.092997:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826309.093368:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef600/0xe21309aa81f4a783 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a78a expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826309.093398:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957380 is committed 00000001:00080000:3.0:1787826309.093409:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957379 is committed 00000004:00080000:3.0:1787826309.093415:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 217 committed 00000100:00100000:2.0:1787826309.095427:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd82850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346198912:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 2482us (2685us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826309.095453:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11080 00010000:00010000:2.0:1787826309.095572:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a7c9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a7c2 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.095595:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5700) 00010000:00010000:2.0:1787826309.095601:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.095618:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.095626:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.095687:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826309.095750:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346199040:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787826309.095776:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346198784:12345-0@lo:103:ldlm_bl.0 Request processed in 12853us (13087us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826309.095788:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21931 00000100:00100000:0.0:1787826309.095857:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.095941:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346199040 00000100:00100000:2.0:1787826309.095955:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346199040 00000100:00080000:0.0:1787826309.096008:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826309.096028:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30819 00000100:00100000:2.0:1787826309.096043:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346199040:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787826309.096073:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346198784:0@lo:103:ldlm_bl.0 00002000:00080000:2.0:1787826309.096096:0:8444:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1311 count 1 00010000:00010000:2.0:1787826309.096200:0:8444:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096222:0:6527:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.096231:0:8444:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 7 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096231:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.096241:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x51f:0x0].0x0 since=6s v=(0 0) 00010000:00010000:2.0:1787826309.096245:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826309.096250:0:8444:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.096289:0:8444:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 7 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.096306:0:8444:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 7 pid: 8444 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.096319:0:8444:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8444:1874663346199168:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826309.096323:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096331:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096338:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096344:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096351:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826309.096356:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096363:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096375:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826309.096378:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826309.096382:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096388:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4a80f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096401:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.096402:0:8444:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00010000:00010000:3.0:1787826309.096407:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096413:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096418:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826309.096421:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096427:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096434:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5700) 00010000:00010000:3.0:1787826309.096436:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826309.096439:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096445:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4a816 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096604:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a808 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013ca expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.096624:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826309.096627:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826309.096631:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4a808 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013ca expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826309.096780:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346198912:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826309.097514:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:3.0:1787826309.097583:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224725:x1874663332584192:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 16661us (19121us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826309.097596:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68395 00000100:00100000:2.0:1787826309.108194:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826309.108235:0:8444:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8444:1874663346199168:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826309.108272:0:8444:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826309.108989:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.109008:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332584320 00000100:00100000:0.0:1787826309.109020:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332584320 00000100:00100000:0.0:1787826309.109093:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68396 00000100:00100000:0.0:1787826309.109129:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224725:x1874663332584320:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826309.109160:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826309.109185:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a824 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013d1 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109265:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109282:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109295:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826309.109358:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826309.109364:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826309.109389:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109419:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a824 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109442:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109453:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826309.109456:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826309.109460:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826309.109531:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224725:x1874663332584320:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 427us (1364us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.109539:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68396 00000100:00100000:0.0:1787826309.109570:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332584448 00000100:00100000:0.0:1787826309.109574:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332584448 00010000:00010000:0.0:1787826309.109608:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 8 pid: 8444 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.109634:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21932 00000100:00100000:0.0:1787826309.109642:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332584448:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826309.109654:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826309.109660:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 8 pid: 8444 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109686:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 8 pid: 8444 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109711:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4a5a0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801329 expref: 8 pid: 8444 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109733:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826309.109739:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826309.109745:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109756:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109785:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826309.109807:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332584448:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 166us (823us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.109815:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21932 00000100:00080000:0.0:1787826309.109834:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787826309.109853:0:8444:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109862:0:8444:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109874:0:8444:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109882:0:8444:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.109907:0:8444:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826309.110140:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346199040:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14101us (14284us total) trans 4294976155 rc 0/0 00000100:00100000:0.0:1787826309.110150:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30819 00010000:00010000:0.0:1787826309.110179:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826309.110187:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.110196:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826309.110202:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.110212:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.110226:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.110234:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000064494fc1/0xe21309aa81f4a81d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.110254:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826309.110256:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826309.110258:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1787826309.110300:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7977c6ea00/2, rc 0, transno 4294976155 00000100:00100000:0.0:1787826309.110308:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346199040:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787826309.117373:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.117484:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332584704 00000100:00100000:0.0:1787826309.117497:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332584704 00000100:00100000:0.0:1787826309.117591:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68397 00000100:00100000:0.0:1787826309.117607:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224725:x1874663332584704:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:0.0:1787826309.117776:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346199296:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826309.117854:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.117895:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00080000:2.0:1787826309.118038:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826309.118143:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346199296 00000100:00100000:3.0:1787826309.118150:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346199296 00000100:00100000:3.0:1787826309.118205:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23612 00000100:00100000:3.0:1787826309.118215:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6529:x1874663346199296:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826309.118314:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6529:x1874663346199296:12345-0@lo:1000:mdt00_002.0 Request processed in 100us (458us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.118323:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23612 00000100:00100000:0.0:1787826309.121930:0:6529:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957380 gen 2 00000004:00080000:0.0:1787826309.121948:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957373 committed transno 8589957380 00000020:00080000:0.0:1787826309.121956:0:6529:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784f7d7b40 refcount 2 committed 0 result 0 batchid 3410 00000020:00080000:0.0:1787826309.121962:0:6529:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f7d7900 00000020:00080000:0.0:1787826309.121966:0:6529:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1513 00000020:00080000:0.0:1787826309.121969:0:6529:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a191c00 00000020:00080000:0.0:1787826309.121972:0:6529:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1195 00000004:00080000:0.0:1787826309.122021:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957374 committed transno 8589957380 00000020:00080000:0.0:1787826309.122025:0:6529:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784f7d76c0 refcount 2 committed 0 result 0 batchid 3411 00000020:00080000:0.0:1787826309.122029:0:6529:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f7d7540 00000020:00080000:0.0:1787826309.122032:0:6529:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1514 00000020:00080000:0.0:1787826309.122034:0:6529:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7846231400 00000020:00080000:0.0:1787826309.122037:0:6529:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1196 00000004:00080000:0.0:1787826309.122061:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957375 committed transno 8589957380 00000004:00080000:0.0:1787826309.122079:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957376 committed transno 8589957380 00000004:00080000:0.0:1787826309.122092:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957377 committed transno 8589957380 00000004:00080000:0.0:1787826309.122140:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957378 committed transno 8589957380 00000004:00080000:0.0:1787826309.122156:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957380 committed transno 8589957380 00000100:00100000:0.0:1787826309.122183:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346199296:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826309.122296:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346199424:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826309.122400:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.122438:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826309.122642:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346199424 00000100:00100000:1.0:1787826309.122651:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346199424 00000100:00100000:1.0:1787826309.122737:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23613 00000100:00100000:1.0:1787826309.122750:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6529:x1874663346199424:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826309.122927:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6529:x1874663346199424:12345-0@lo:1000:mdt00_002.0 Request processed in 178us (526us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826309.122935:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23613 00000100:00100000:0.0:1787826309.123060:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346199424:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1787826309.123235:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.123248:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.123266:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.123282:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346199552:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826309.123313:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.123346:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826309.123448:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346199552 00000100:00100000:2.0:1787826309.123455:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346199552 00000100:00100000:2.0:1787826309.123493:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23614 00000100:00100000:2.0:1787826309.123502:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6529:x1874663346199552:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787826309.123515:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826309.123537:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a832 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.123570:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.123580:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.123590:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.123620:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.123636:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 6 pid: 24702 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.123648:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346199680:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826309.123757:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.123768:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 6 pid: 35093 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.123777:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346199808:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826309.123800:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4180 going to sleep for 11 seconds 00000020:00080000:3.0:1787826309.124636:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3411 committed batchid 3409 00000020:00080000:3.0:1787826309.124733:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3409 00000100:00100000:3.0:1787826309.127000:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826309.127740:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332584832 00000100:00100000:1.0:1787826309.127758:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332584832 00010000:00010000:1.0:1787826309.127821:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 7 pid: 35093 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826309.127879:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21933 00000100:00100000:1.0:1787826309.127893:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332584832:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826309.127911:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826309.127919:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 7 pid: 35093 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.127934:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 7 pid: 35093 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.127961:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826309.127969:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.127980:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826309.128052:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332584832:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 162us (1065us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826309.128060:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21933 00000100:00100000:0.0:1787826309.128810:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346199680:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826309.128830:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a801 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013c3 expref: 6 pid: 35093 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.128851:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346199808:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826309.128888:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.128899:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826309.128902:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826309.128907:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.128917:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:0.0:1787826309.128985:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346199552:12345-0@lo:101:mdt00_002.0 Request processed in 5483us (5670us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.128994:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23614 00000100:00100000:1.0:1787826309.129086:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346199552:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826309.129136:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.129149:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826309.129709:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.130189:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332584960 00000100:00100000:0.0:1787826309.130200:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332584960 00010000:00010000:0.0:1787826309.130243:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 7 pid: 24702 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.130340:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21934 00000100:00100000:0.0:1787826309.130352:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332584960:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826309.130371:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826309.130381:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 7 pid: 24702 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.130392:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 7 pid: 24702 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.130417:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a82b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608013d1 expref: 7 pid: 24702 timeout: 8221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.130434:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826309.130440:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.130457:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.130466:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.130511:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.130528:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346199936:0@lo:105: 00000100:00100000:0.0:1787826309.130590:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.130644:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826309.130973:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346199936 00000100:00100000:3.0:1787826309.130980:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346199936 00000100:00100000:3.0:1787826309.131031:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7857 00000100:00100000:3.0:1787826309.131041:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346199936:12345-0@lo:105: 00010000:00010000:3.0:1787826309.131054:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.131065:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.131082:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.131090:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.131098:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.131133:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.131158:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42000) 00000100:00100000:3.0:1787826309.131231:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346199936:12345-0@lo:105: Request processed in 191us (639us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.131240:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7857 00000100:00100000:1.0:1787826309.131470:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346199936:0@lo:105: 00010000:00010000:1.0:1787826309.131497:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826309.131559:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332584960:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1208us (1853us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826309.131567:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21934 00010000:00010000:0.0:1787826309.131650:0:6529:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.131672:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826309.131700:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826309.131752:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.131761:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.131774:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.131794:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200064:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826309.131836:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.131866:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826309.132942:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346200064 00000100:00100000:3.0:1787826309.132958:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346200064 00000100:00100000:3.0:1787826309.133031:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23615 00000100:00100000:3.0:1787826309.133044:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346200064:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787826309.133071:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826309.133099:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a847 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a840 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.133160:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a847 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a840 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.133407:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a847 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a840 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.133422:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7843326000) 00010000:00010000:3.0:1787826309.133425:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826309.133429:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826309.133522:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200064:12345-0@lo:101:mdt00_002.0 Request processed in 481us (1684us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.133531:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23615 00000100:00100000:0.0:1787826309.133772:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200064:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787826309.133796:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.133809:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826309.133945:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826309.134018:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200192:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826309.134073:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.134159:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826309.134813:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346200192 00000100:00100000:1.0:1787826309.134825:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346200192 00000100:00100000:1.0:1787826309.134887:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23616 00000100:00100000:1.0:1787826309.134900:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200192:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826309.135024:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200192:12345-0@lo:1000:mdt00_002.0 Request processed in 126us (950us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826309.135032:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23616 00000100:00100000:2.0:1787826309.139785:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200192:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.140303:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200320:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.140643:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.140704:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826309.140782:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346200320 00000100:00100000:0.0:1787826309.140793:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346200320 00000100:00100000:0.0:1787826309.141043:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23617 00000100:00100000:0.0:1787826309.141057:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200320:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826309.141260:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200320:12345-0@lo:1000:mdt00_002.0 Request processed in 205us (616us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826309.141268:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23617 00000100:00100000:2.0:1787826309.144424:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200320:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.144611:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200448:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.144701:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.144753:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826309.144764:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346200448 00000100:00100000:3.0:1787826309.144772:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346200448 00000100:00100000:3.0:1787826309.144848:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23618 00000100:00100000:3.0:1787826309.144861:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200448:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826309.145012:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200448:12345-0@lo:1000:mdt00_002.0 Request processed in 153us (310us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787826309.145020:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23618 00000100:00100000:2.0:1787826309.145213:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200448:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.145271:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200576:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.145297:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.145323:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826309.145377:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346200576 00000100:00100000:3.0:1787826309.145382:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346200576 00000100:00100000:3.0:1787826309.145413:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23619 00000100:00100000:3.0:1787826309.145421:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200576:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826309.145486:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200576:12345-0@lo:1000:mdt00_002.0 Request processed in 65us (186us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.145492:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23619 00000100:00100000:2.0:1787826309.145642:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200576:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.145697:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200704:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.145721:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.145743:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826309.146740:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346200704 00000100:00100000:0.0:1787826309.146753:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346200704 00000100:00100000:1.0:1787826309.147052:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23620 00000100:00100000:1.0:1787826309.147064:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200704:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826309.147253:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200704:12345-0@lo:1000:mdt00_002.0 Request processed in 190us (1530us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826309.147261:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23620 00000100:00100000:0.0:1787826309.147349:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200704:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826309.147446:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf2aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200832:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826309.147488:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.149254:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826309.149698:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346200832 00000100:00100000:2.0:1787826309.149712:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346200832 00000100:00100000:2.0:1787826309.149793:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23621 00000100:00100000:2.0:1787826309.149812:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200832:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826309.149960:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346200832:12345-0@lo:1000:mdt00_002.0 Request processed in 158us (2472us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826309.149968:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23621 00000100:00100000:0.0:1787826309.151358:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf2aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346200832:0@lo:1000:mdt00_002.0 00000040:00080000:0.0:1787826309.152493:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1198, 2168 off1091728 00000040:00080000:0.0:1787826309.152591:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1516, 2368 off1533152 00000004:00080000:0.0:1787826309.152599:0:6529:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784f7d7000 with version 6827 00000004:00080000:2.0:1787826309.152915:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6827 rpc_version 6827 00000020:00100000:2.0:1787826309.152970:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826309.152982:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7848379180 x1874663346201088/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826309.153010:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346201088:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826309.153133:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.153170:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826309.153878:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346201088 00000100:00100000:1.0:1787826309.153891:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346201088 00000100:00100000:1.0:1787826309.154042:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23622 00000100:00100000:1.0:1787826309.154058:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346201088:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826309.155208:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346201088:12345-0@lo:1000:osp_up0-1.0 Request processed in 1148us (2069us total) trans 8589957381 rc 0/0 00000100:00100000:1.0:1787826309.155221:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23622 00000100:00100000:0.0:1787826309.155382:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346201088:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826309.155525:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.155548:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.155573:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.155603:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.155610:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.155616:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.155652:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.155659:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.155678:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.155697:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf2aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346201216:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826309.155740:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.155998:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826309.156232:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346201216 00000100:00100000:2.0:1787826309.156242:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346201216 00000100:00100000:2.0:1787826309.156289:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23623 00000100:00100000:2.0:1787826309.156298:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346201216:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787826309.156320:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826309.156340:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a863 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a85c expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.156365:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a863 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a85c expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.156378:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a863 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a85c expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.156388:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:2.0:1787826309.156391:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826309.156396:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826309.156442:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6529:x1874663346201216:12345-0@lo:101:mdt00_002.0 Request processed in 145us (700us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826309.156449:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23623 00000100:00100000:1.0:1787826309.156547:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf2aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346201216:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826309.156630:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156643:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826309.156679:0:6529:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000022587bf2 reply state 00000000e183c978 transno 4294985391 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156694:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156699:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:1.0:1787826309.156701:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826309.156703:0:6529:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826309.156708:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156714:0:6529:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826309.156722:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x192:0x0] 00000004:00010000:1.0:1787826309.156726:0:6529:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000022587bf2 reply state 00000000e183c978 transno 4294985391 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156735:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156739:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:1.0:1787826309.156740:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826309.156741:0:6529:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787826309.156747:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4c:0x0]: returning size 4096 00000004:00080000:1.0:1787826309.156755:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826309.156763:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156767:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156771:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156789:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826309.156793:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826309.156797:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.156822:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826309.156830:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.156838:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826309.156845:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826309.156853:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.156875:0:6529:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.156883:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.156889:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826309.157002:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346201344:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826309.157055:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826309.157157:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.157166:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4a832 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4a839 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.157182:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826309.157185:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826309.157186:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826309.157197:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224725:x1874663332584704:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 39592us (39837us total) trans 4294985391 rc 0/0 00000100:00100000:0.0:1787826309.157199:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346201344 00000100:00100000:0.0:1787826309.157205:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346201344 00000100:00100000:1.0:1787826309.157205:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68397 00000100:00100000:0.0:1787826309.157252:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11081 00000100:00100000:0.0:1787826309.157260:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346201344:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826309.157458:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826309.157482:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4a839 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a832 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.157496:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826309.157500:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826309.157542:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346201344:12345-0@lo:103:ldlm_bl.0 Request processed in 283us (487us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.157550:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11081 00000100:00100000:2.0:1787826309.157741:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346201344:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826309.166357:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826309.166755:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332585088 00000100:00100000:2.0:1787826309.166768:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332585088 00000100:00100000:2.0:1787826309.166983:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68398 00000100:00100000:2.0:1787826309.166998:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224714:x1874663332585088:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826309.167025:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826309.167050:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a86a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013d8 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.167200:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a871 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.167223:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a871 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.167237:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a871 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826309.167300:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826309.167304:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826309.167328:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a871 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.167359:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4a86a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013d8 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.167382:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a871 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013d8 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.167391:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826309.167395:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826309.167399:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826309.167479:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224714:x1874663332585088:12345-192.168.203.56@tcp:101:bash.0 Request processed in 485us (1141us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826309.167487:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68398 00000100:00100000:3.0:1787826309.171617:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.172090:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332585216 00000100:00100000:0.0:1787826309.172102:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332585216 00000100:00100000:0.0:1787826309.172303:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68399 00000100:00100000:0.0:1787826309.172315:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224714:x1874663332585216:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826309.172341:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826309.172368:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a878 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013df expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.172450:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.172466:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.172479:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.172491:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.172507:0:35098:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a847 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a840 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.172517:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.172560:0:35098:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a847 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a840 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.172578:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000_UUID:35098:1874663346201472:0@lo:104: 00000100:00100000:0.0:1787826309.173032:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.173194:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826309.174000:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346201472 00000100:00100000:3.0:1787826309.174012:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346201472 00000100:00100000:3.0:1787826309.174140:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7858 00000100:00100000:3.0:1787826309.174153:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35098:x1874663346201472:12345-0@lo:104: 00010000:00010000:3.0:1787826309.174174:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.174284:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826309.174291:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35098:x1874663346201472:12345-0@lo:104: Request processed in 141us (1262us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.174298:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7858 00010000:00010000:2.0:1787826309.174367:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826309.174375:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.174385:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826309.174392:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826309.174403:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.174444:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.174450:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826309.174475:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.174484:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4a840 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a847 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.174506:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826309.174508:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826309.174510:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826309.174517:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826309.174531:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346201600:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826309.174573:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.174772:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000_UUID:35098:1874663346201472:0@lo:104: 00010000:00010000:0.0:1787826309.174800:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.175296:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346201600 00000100:00100000:2.0:1787826309.175304:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346201600 00010000:00010000:2.0:1787826309.175339:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a847 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a840 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.175382:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21935 00000100:00100000:2.0:1787826309.175393:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346201600:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826309.175411:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787826309.180592:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957381 is committed 00010000:00010000:0.0:1787826309.180823:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4a847 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a840 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.180849:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7843326000) 00010000:00010000:0.0:1787826309.180855:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.180865:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.180873:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.180921:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826309.181034:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346201600:12345-0@lo:103:ldlm_bl.0 Request processed in 5640us (6457us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.181046:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21935 00010000:00010000:3.0:1787826309.181414:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.181424:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826309.181595:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-mdtlov_UUID:35098:1874663346201728:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826309.181741:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826309.181824:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826309.181942:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346201600:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826309.182477:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346201728 00000100:00100000:0.0:1787826309.182488:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346201728 00000100:00100000:0.0:1787826309.182593:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23624 00000100:00100000:0.0:1787826309.182605:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:35098:x1874663346201728:12345-0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826309.182757:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:35098:x1874663346201728:12345-0@lo:1000:mdt00_013.0 Request processed in 151us (1015us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.182765:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23624 00000100:00100000:3.0:1787826309.182843:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-mdtlov_UUID:35098:1874663346201728:0@lo:1000:mdt00_013.0 00010000:00010000:3.0:1787826309.182909:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a88d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.182928:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a88d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.182940:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a88d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826309.182972:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a88d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.182990:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.182997:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183004:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183010:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183017:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826309.183021:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183029:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183043:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826309.183046:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826309.183051:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183057:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4a87f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183073:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183081:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183086:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183091:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826309.183093:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183099:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183138:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7843326000) 00010000:00010000:3.0:1787826309.183141:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826309.183144:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183149:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4a886 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183469:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4a878 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013df expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183489:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a88d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013df expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.183497:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f79460a2300) 00010000:00010000:3.0:1787826309.183500:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826309.183504:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826309.183577:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224714:x1874663332585216:12345-192.168.203.56@tcp:101:bash.0 Request processed in 11263us (11973us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.183586:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68399 00000100:00100000:3.0:1787826309.185945:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.186030:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332585344 00000100:00100000:0.0:1787826309.186043:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332585344 00000100:00100000:0.0:1787826309.186148:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68400 00000100:00100000:0.0:1787826309.186161:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224714:x1874663332585344:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826309.186194:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826309.186220:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a894 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013e6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.186303:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a89b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.186318:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a89b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.186331:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a89b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826309.186451:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4c:0x0]: returning size 4096 00000004:00200000:0.0:1787826309.186457:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826309.186499:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4a894 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013e6 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.186521:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a89b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013e6 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.186530:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:0.0:1787826309.186533:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826309.186536:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826309.187182:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224714:x1874663332585344:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1018us (1242us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.187192:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68400 00000100:00100000:2.0:1787826309.194378:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826309.194488:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332585472 00000100:00100000:3.0:1787826309.194504:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332585472 00000100:00100000:3.0:1787826309.194586:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68401 00000100:00100000:3.0:1787826309.194600:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224714:x1874663332585472:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826309.194626:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826309.194893:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4a8a2 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013ed expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.194985:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4a8a9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.195003:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4a8a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.195016:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4a8a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826309.195083:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x194:0x0]: returning size 4096 00000004:00200000:3.0:1787826309.195089:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826309.195400:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4a8a2 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013ed expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.195425:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4a8a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013ed expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.195440:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:3.0:1787826309.195443:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826309.195448:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826309.195523:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224714:x1874663332585472:12345-192.168.203.56@tcp:101:bash.0 Request processed in 927us (1162us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.195532:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68401 00000100:00100000:3.0:1787826309.199921:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.203356:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332585600 00000100:00100000:0.0:1787826309.203377:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332585600 00000100:00100000:1.0:1787826309.206705:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68402 00000100:00100000:1.0:1787826309.206729:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224714:x1874663332585600:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826309.206763:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826309.206800:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a8b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013f4 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.206885:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.206904:0:11629:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a863 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a85c expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.206913:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.206974:0:11629:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a863 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4a85c expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826309.206994:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346201856:0@lo:104: 00000100:00100000:1.0:1787826309.207079:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826309.207225:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78555e7600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826309.207639:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346201856 00000100:00100000:2.0:1787826309.207650:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346201856 00000100:00100000:2.0:1787826309.207751:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7859 00000100:00100000:2.0:1787826309.207762:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346201856:12345-0@lo:104: 00010000:00010000:2.0:1787826309.207779:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.207875:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826309.207881:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346201856:12345-0@lo:104: Request processed in 122us (804us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826309.207889:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7859 00000100:00100000:3.0:1787826309.207969:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346201856:0@lo:104: 00010000:00010000:3.0:1787826309.208002:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.208072:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826309.208081:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.208091:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826309.208097:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826309.208147:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.208198:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.208206:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826309.208229:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 1/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.208237:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4a85c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4a863 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.208271:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826309.208274:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826309.208275:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826309.208285:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787826309.208300:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346201984:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826309.208344:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826309.208382:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346201984 00000100:00100000:0.0:1787826309.208387:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346201984 00010000:00010000:0.0:1787826309.208411:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a863 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4a85c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826309.208437:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21936 00000100:00100000:0.0:1787826309.208444:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346201984:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826309.208455:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826309.209550:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4a863 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4a85c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.209576:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:0.0:1787826309.209582:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.209593:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.209601:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.209630:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826309.209702:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346201984:12345-0@lo:103:ldlm_bl.0 Request processed in 1258us (1357us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.209711:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21936 00000100:00100000:0.0:1787826309.209768:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346201984:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826309.209798:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.209807:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826309.209889:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x192:0x0]: returning size 4096 00000004:00200000:0.0:1787826309.209896:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826309.214085:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4a8b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013f4 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.214157:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013f4 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.214176:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:0.0:1787826309.214180:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826309.214538:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:0.0:1787826309.214651:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224714:x1874663332585600:12345-192.168.203.56@tcp:101:bash.0 Request processed in 7925us (14742us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.214672:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68402 00000100:00100000:1.0:1787826309.219147:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826309.219503:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332585856 00000100:00100000:3.0:1787826309.219517:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332585856 00000100:00100000:3.0:1787826309.219597:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68403 00000100:00100000:3.0:1787826309.219610:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224714:x1874663332585856:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826309.219639:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826309.219643:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826309.219690:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4a8b7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013f4 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.219718:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:3.0:1787826309.219737:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826309.219757:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a8be lrc: 2/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608013fb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.219929:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.219956:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.219967:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.219982:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 3/0,1 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.219989:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.219995:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826309.221640:0:17813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x51f:0x0], index 0: 0 00010000:00010000:3.0:1787826309.221865:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a8d3 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221881:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a8d3 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221892:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a8d3 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221937:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221945:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221952:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221961:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221967:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826309.221972:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221980:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.221993:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826309.221996:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826309.222001:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.222007:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a8c5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826309.222022:0:17813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dc88743d reply state 000000005c23613a transno 8589957382 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.222035:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.222041:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826309.222043:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826309.222047:0:17813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826309.222088:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a8be lrc: 0/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013fb expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.222139:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a8d3 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013fb expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.222147:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:3.0:1787826309.222150:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826309.222152:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826309.224904:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224714:x1874663332585856:12345-192.168.203.56@tcp:101:bash.0 Request processed in 5278us (5754us total) trans 8589957382 rc 0/0 00000100:00100000:3.0:1787826309.224921:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68403 00000100:00100000:3.0:1787826309.227836:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826309.227928:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332585984 00000100:00100000:1.0:1787826309.227948:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332585984 00000100:00100000:1.0:1787826309.228045:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30820 00000100:00100000:1.0:1787826309.228059:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:224714:x1874663332585984:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826309.228085:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826309.228139:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801402 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826309.228175:0:8445:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x51f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787826309.228217:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=8121s v=(0 0) 00010000:00010000:1.0:1787826309.228223:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826309.228236:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.228256:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.228274:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826309.228350:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224714:x1874663332585984:12345-192.168.203.56@tcp:101:bash.0 Request processed in 295us (529us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826309.228358:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30820 00000100:00080000:2.0:1787826309.228881:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826309.230912:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826309.231019:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332586240 00000100:00100000:2.0:1787826309.231032:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332586240 00000100:00100000:2.0:1787826309.231142:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9338 00000100:00100000:2.0:1787826309.231164:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224714:x1874663332586240:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826309.231964:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224714:x1874663332586240:12345-192.168.203.56@tcp:35:bash.0 Request processed in 808us (1060us total) trans 8589957383 rc 0/0 00000100:00100000:2.0:1787826309.231975:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9338 00000100:00100000:1.0:1787826309.237056:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826309.237138:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332586496 00000100:00100000:2.0:1787826309.237152:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332586496 00000100:00100000:2.0:1787826309.237230:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68404 00000100:00100000:2.0:1787826309.237243:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224714:x1874663332586496:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826309.237284:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826309.237290:0:35080:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826309.237328:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4a8a9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013ed expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.237350:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:2.0:1787826309.237356:0:35080:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826309.237373:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4a8e8 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801409 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.237534:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 3/0,1 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.237549:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.237562:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.237581:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.237588:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.237595:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.237774:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fd880 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346202112:0@lo:1000:mdt00_008.0 00000100:00100000:2.0:1787826309.237855:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826309.237899:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826309.237984:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346202112 00000100:00100000:3.0:1787826309.237991:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346202112 00000100:00100000:3.0:1787826309.238030:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23625 00000100:00100000:3.0:1787826309.238039:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:35080:x1874663346202112:12345-0@lo:1000:mdt00_008.0 00000100:00100000:3.0:1787826309.238155:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:35080:x1874663346202112:12345-0@lo:1000:mdt00_008.0 Request processed in 115us (299us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.238163:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23625 00000100:00100000:2.0:1787826309.238228:0:35080:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957381 gen 2 00000004:00080000:2.0:1787826309.238235:0:35080:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957381 committed transno 8589957381 00000100:00100000:2.0:1787826309.238362:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fd880 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346202112:0@lo:1000:mdt00_008.0 00000004:00080000:2.0:1787826309.238711:0:35080:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4ca:0x0], index 1: 0 00010000:00010000:2.0:1787826309.238899:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a8fd lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.238911:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a8fd lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.238921:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a8fd lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239360:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239369:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239376:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239385:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239392:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826309.239397:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239405:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239419:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:2.0:1787826309.239423:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826309.239428:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 3/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239434:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4a8ef lrc: 0/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826309.239450:0:35080:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ae7e6279 reply state 000000005bd55fa2 transno 4294985392 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239462:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239468:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x4ec94fd6 (ffff9f7952ca9300) 00010000:00010000:2.0:1787826309.239471:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826309.239474:0:35080:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826309.239512:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4a8e8 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801409 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239528:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a8fd lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801409 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.239566:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f7952ca9500) 00010000:00010000:2.0:1787826309.239570:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826309.239573:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826309.239944:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224714:x1874663332586496:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2692us (2891us total) trans 4294985392 rc 0/0 00000100:00100000:2.0:1787826309.239954:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68404 00000100:00100000:3.0:1787826309.243808:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.244585:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332586624 00000100:00100000:0.0:1787826309.244601:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332586624 00000100:00100000:0.0:1787826309.244705:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30821 00000100:00100000:0.0:1787826309.244720:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224714:x1874663332586624:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826309.244748:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826309.244775:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801410 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826309.244807:0:35096:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4ca:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787826309.244839:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ca:0x0].0x0 since=8121s v=(0 0) 00010000:00010000:0.0:1787826309.244845:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ca:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826309.244860:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 8 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.244881:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 8 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826309.244912:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826309.245001:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224714:x1874663332586624:12345-192.168.203.56@tcp:101:bash.0 Request processed in 284us (1209us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826309.245009:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30821 00000100:00100000:0.0:1787826309.248298:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826309.248378:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332586880 00000100:00100000:2.0:1787826309.248392:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332586880 00000100:00100000:2.0:1787826309.248472:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9339 00000100:00100000:2.0:1787826309.248485:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224714:x1874663332586880:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826309.248886:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224714:x1874663332586880:12345-192.168.203.56@tcp:35:bash.0 Request processed in 398us (598us total) trans 4294985393 rc 0/0 00000100:00100000:2.0:1787826309.248896:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9339 00000100:00100000:1.0:1787826309.279378:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826309.279433:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826309.279468:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826309.279557:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332587008 00000100:00100000:3.0:1787826309.279568:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332587008 00000100:00100000:3.0:1787826309.279643:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332587136 00000100:00100000:3.0:1787826309.279645:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332587136 00000100:00100000:3.0:1787826309.279676:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332587264 00000100:00100000:3.0:1787826309.279678:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332587264 00000100:00100000:3.0:1787826309.279703:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11082 00000100:00100000:3.0:1787826309.279714:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971595450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:34879:x1874663332587008:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826309.279734:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826309.279763:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4a8fd lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801409 expref: 13 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.279788:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f7952ca9500) 00010000:00010000:3.0:1787826309.279805:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4a89b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013e6 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.279816:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:3.0:1787826309.279823:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4a648 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801368 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.279830:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4a:0x0].0x0 (ffff9f784b4a5a00) 00010000:00010000:3.0:1787826309.279837:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826309.279896:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971595450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:34879:x1874663332587008:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 183us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.279903:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11082 00000100:00100000:3.0:1787826309.279955:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11083 00000100:00100000:3.0:1787826309.279961:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:15672:x1874663332587136:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826309.279969:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826309.279981:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4a871 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013d8 expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.279993:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826309.280001:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4a8d3 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013fb expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.280012:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:3.0:1787826309.280022:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a88d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608013df expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826309.280029:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f79460a2300) 00010000:00010000:3.0:1787826309.280034:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826309.280048:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:15672:x1874663332587136:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 87us (615us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.280054:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11083 00000100:00100000:1.0:1787826309.280173:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11084 00000100:00100000:1.0:1787826309.280184:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:49149:x1874663332587264:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826309.280193:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826309.280208:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4a6f0 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080138b expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826309.280225:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4b:0x0].0x0 (ffff9f7843326700) 00010000:00010000:1.0:1787826309.280233:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826309.280252:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:49149:x1874663332587264:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 72us (783us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826309.280258:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11084 00010000:00010000:0.0:1787826309.296076:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826309.341515:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.341566:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.341615:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826309.341728:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332587392 00000100:00100000:1.0:1787826309.341742:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332587392 00000100:00100000:1.0:1787826309.341814:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2220 00000100:00100000:1.0:1787826309.341827:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf41850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332587392:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826309.341919:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf41850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332587392:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 94us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826309.341929:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2220 00000100:00100000:3.0:1787826309.341944:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332587648 00000100:00100000:3.0:1787826309.341949:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332587648 00000100:00100000:3.0:1787826309.341993:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332587520 00000100:00100000:3.0:1787826309.341997:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332587520 00000100:00100000:3.0:1787826309.342021:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8806 00000100:00100000:3.0:1787826309.342027:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332587648:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826309.342061:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332587648:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 34us (495us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.342068:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8806 00000100:00100000:3.0:1787826309.342082:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8807 00000100:00100000:3.0:1787826309.342089:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796eda6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332587520:12345-192.168.203.56@tcp:400:kworker.0 00000100:00080000:2.0:1787826309.342697:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826309.342703:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826309.347871:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796eda6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332587520:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 5760us (6233us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.347892:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8807 00000100:00100000:3.0:1787826309.855727:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826309.855777:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.855868:0:24157:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332587776 00000100:00100000:0.0:1787826309.855881:0:24157:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332587776 00002000:00100000:0.0:1787826309.855957:0:24157:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x51f:0x0] (0->5) req@ffff9f785124ea00 x1874663332587776/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:665/0 lens 496/0 e 0 to 0 dl 1787826320 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826309.855970:0:24157:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124ea00 with x1874663332587776 ext(0->5) 00002000:00010000:0.0:1787826309.855977:0:24157:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f785124ea00 00000100:00100000:0.0:1787826309.856014:0:24157:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27645 00000100:00100000:0.0:1787826309.856026:0:24157:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332587776:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:0.0:1787826309.858447:0:24157:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332587776:12345-192.168.203.56@tcp:4:bash.0 Request processed in 2409us (2721us total) trans 4294973847 rc 0/0 00000100:00100000:0.0:1787826309.858463:0:24157:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27645 00002000:00100000:0.0:1787826309.858480:0:24157:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x51f:0x0] (0->5) req@ffff9f785124ea00 x1874663332587776/t4294973847(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:665/0 lens 496/448 e 0 to 0 dl 1787826320 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826309.858491:0:24157:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124ea00 with x1874663332587776 ext(0->5) 00002000:00010000:0.0:1787826309.858498:0:24157:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f785124ea00 00000100:00100000:2.0:1787826309.866773:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332587904 00000100:00100000:2.0:1787826309.866796:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332587904 00000100:00100000:3.0:1787826309.867152:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30822 00000100:00100000:3.0:1787826309.867167:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332587904:12345-192.168.203.56@tcp:16:kworker.0 00000001:00080000:2.0:1787826309.881192:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973847 is committed 00000001:00080000:2.0:1787826309.881221:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973846 is committed 00000100:00100000:3.0:1787826309.881483:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332587904:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 14311us (25695us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826309.881497:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30822 00000100:00100000:3.0:1787826309.884269:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826309.884328:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826309.884481:0:8450:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332588032 00000100:00100000:0.0:1787826309.884493:0:8450:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332588032 00002000:00100000:0.0:1787826309.884559:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x4ca:0x0] (0->5) req@ffff9f785124c700 x1874663332588032/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:665/0 lens 496/0 e 0 to 0 dl 1787826320 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826309.884571:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124c700 with x1874663332588032 ext(0->5) 00002000:00010000:0.0:1787826309.884578:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f785124c700 00000100:00100000:0.0:1787826309.884606:0:8450:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27646 00000100:00100000:0.0:1787826309.884616:0:8450:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332588032:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826309.886469:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332588160 00000100:00100000:2.0:1787826309.886488:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332588160 00000100:00100000:2.0:1787826309.886577:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30823 00000100:00100000:2.0:1787826309.886588:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332588160:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:2.0:1787826309.886736:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332588160:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 148us (2480us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826309.886749:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30823 00000100:00100000:0.0:1787826309.898043:0:8450:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2395:x1874663332588032:12345-192.168.203.56@tcp:4:bash.0 Request processed in 13416us (13703us total) trans 4294976156 rc 0/0 00000100:00100000:0.0:1787826309.898059:0:8450:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27646 00002000:00100000:0.0:1787826309.898073:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x4ca:0x0] (0->5) req@ffff9f785124c700 x1874663332588032/t4294976156(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:665/0 lens 496/448 e 0 to 0 dl 1787826320 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826309.898083:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124c700 with x1874663332588032 ext(0->5) 00002000:00010000:0.0:1787826309.898090:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f785124c700 00000001:00080000:2.0:1787826309.904459:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976156 is committed 00000001:00080000:2.0:1787826309.904489:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976155 is committed 00000001:00080000:2.0:1787826309.904496:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976154 is committed 00000100:00100000:2.0:1787826310.126345:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826310.127056:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346202368:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.127186:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826310.127229:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346202880:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.127256:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826310.127279:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346203392:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.127300:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826310.127556:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346202368 00000100:00100000:0.0:1787826310.127564:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346202368 00000100:00100000:1.0:1787826310.127596:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1787826310.127607:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346202496:0@lo:400:kworker.0 00000100:00100000:0.0:1787826310.127613:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346203392 00000100:00100000:0.0:1787826310.127618:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346203392 00000100:00100000:0.0:1787826310.127646:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23626 00000100:00100000:0.0:1787826310.127669:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346202368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826310.127735:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346202368:12345-0@lo:400:kworker.0 Request processed in 80us (550us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826310.127742:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23626 00000100:00100000:0.0:1787826310.127763:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23627 00000100:00100000:0.0:1787826310.127770:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346203392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826310.130564:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346202624:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.133031:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826310.133091:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346203392:12345-0@lo:400:kworker.0 Request processed in 5318us (5786us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826310.133130:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23627 00000100:00100000:3.0:1787826310.133176:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346202880 00000100:00100000:3.0:1787826310.133187:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346202880 00000100:00080000:0.0:1787826310.133231:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826310.133251:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8808 00000100:00100000:3.0:1787826310.133264:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346202880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.133738:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346203008:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.133783:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826310.133815:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346203520:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.133842:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826310.133894:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346202368:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.133909:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346203392:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.133961:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346202496 00000100:00100000:1.0:1787826310.133966:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346202496 00000100:00100000:1.0:1787826310.134000:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346203008 00000100:00100000:1.0:1787826310.134003:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346203008 00000100:00100000:1.0:1787826310.134031:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14284 00000100:00100000:1.0:1787826310.134039:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346202496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.134095:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346202496:12345-0@lo:400:kworker.0 Request processed in 56us (1073us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826310.134102:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14284 00000100:00100000:1.0:1787826310.134145:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14285 00000100:00100000:1.0:1787826310.134153:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346203008:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.134184:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346203008:12345-0@lo:400:kworker.0 Request processed in 33us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826310.134191:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14285 00000100:00100000:1.0:1787826310.134209:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346203520 00000100:00100000:1.0:1787826310.134212:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346203520 00000100:00100000:1.0:1787826310.134239:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8809 00000100:00100000:1.0:1787826310.134245:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346203520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.134275:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346203520:12345-0@lo:400:kworker.0 Request processed in 30us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826310.134281:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8809 00000100:00080000:1.0:1787826310.134306:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826310.134316:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826310.134328:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346202752:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.134354:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826310.134407:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346203264:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.134430:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826310.134457:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346203776:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.134481:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826310.134922:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346203264 00000100:00100000:2.0:1787826310.134937:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346203264 00000100:00100000:2.0:1787826310.135016:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8810 00000100:00100000:2.0:1787826310.135027:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346203264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826310.135133:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346203264:12345-0@lo:400:kworker.0 Request processed in 107us (700us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826310.135140:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8810 00000100:00100000:1.0:1787826310.135632:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976154 gen 2 00000004:00080000:1.0:1787826310.135642:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7854627480, transno 4294976154 00000100:00100000:1.0:1787826310.135672:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346203264:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.135711:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346202752 00000100:00100000:1.0:1787826310.135717:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346202752 00000100:00100000:1.0:1787826310.135760:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346203776 00000100:00100000:1.0:1787826310.135763:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346203776 00000100:00100000:1.0:1787826310.135785:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14286 00000100:00100000:1.0:1787826310.135794:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346202752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.135854:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346202752:12345-0@lo:400:kworker.0 Request processed in 62us (1499us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826310.135861:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14286 00000100:00100000:1.0:1787826310.135876:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14287 00000100:00100000:1.0:1787826310.135882:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346203776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.135909:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346203776:12345-0@lo:400:kworker.0 Request processed in 28us (1428us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826310.135915:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14287 00000100:00100000:0.0:1787826310.136275:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346202496:0@lo:400:kworker.0 00000100:00100000:0.0:1787826310.136289:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346203008:0@lo:400:kworker.0 00000100:00100000:0.0:1787826310.136311:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346203520:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.136860:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346202752:0@lo:400:kworker.0 00000100:00100000:1.0:1787826310.136984:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346203776:0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.137239:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346202880:12345-0@lo:400:kworker.0 Request processed in 3977us (9979us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826310.137249:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8808 00000100:00080000:3.0:1787826310.137311:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826310.137316:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826310.137364:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826310.137415:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346202624 00000100:00100000:3.0:1787826310.137421:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346202624 00000100:00100000:3.0:1787826310.137463:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14288 00000100:00100000:3.0:1787826310.137471:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346202624:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.137537:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346202624:12345-0@lo:400:kworker.0 Request processed in 67us (173us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826310.137543:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14288 00000100:00080000:3.0:1787826310.137557:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826310.137576:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346202624:0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.137585:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346203136:0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.137613:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826310.137632:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346203648:0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.137668:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826310.137690:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346202240:0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.137720:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826310.137772:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346203648 00000100:00100000:3.0:1787826310.137776:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346203648 00000100:00100000:3.0:1787826310.137801:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8811 00000100:00100000:3.0:1787826310.137806:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346203648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.137839:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346203648:12345-0@lo:400:kworker.0 Request processed in 33us (181us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826310.137844:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8811 00000100:00100000:3.0:1787826310.137869:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976155 gen 2 00000004:00080000:3.0:1787826310.137883:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7977c6ea00, transno 4294976155 00000100:00100000:3.0:1787826310.137893:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346203648:0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.138031:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346202240 00000100:00100000:3.0:1787826310.138035:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346202240 00000100:00100000:3.0:1787826310.138064:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2221 00000100:00100000:3.0:1787826310.138070:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7974f69850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346202240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.138131:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7974f69850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346202240:12345-0@lo:400:kworker.0 Request processed in 35us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826310.138139:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2221 00000100:00100000:3.0:1787826310.138165:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346203136 00000100:00100000:3.0:1787826310.138169:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346203136 00000100:00100000:3.0:1787826310.138201:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14289 00000100:00100000:3.0:1787826310.138208:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346203136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.138237:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346203136:12345-0@lo:400:kworker.0 Request processed in 30us (624us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826310.138242:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14289 00000100:00080000:3.0:1787826310.138262:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826310.138279:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346203136:0@lo:400:kworker.0 00000100:00100000:3.0:1787826310.138287:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346202240:0@lo:400:kworker.0 00000100:00100000:0.0:1787826310.138857:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973846 gen 2 00000004:00080000:0.0:1787826310.138865:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7977e0c700, transno 4294973846 00000100:00100000:0.0:1787826310.138987:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346202880:0@lo:400:kworker.0 00010000:00010000:0.0:1787826310.319401:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826310.638676:0:165857:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:2.0:1787826310.638722:0:165857:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000100:00100000:1.0:1787826310.646603:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826310.646628:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346203904:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826310.646727:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826310.647441:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346203904 00000100:00100000:3.0:1787826310.647451:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346203904 00000100:00100000:2.0:1787826310.654256:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6223 00000100:00100000:2.0:1787826310.654278:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346203904:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1787826310.654311:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683135488 00000100:00100000:2.0:1787826310.654423:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346203904:12345-0@lo:13:osp-pre-1-1.0 Request processed in 154us (7695us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826310.654430:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6223 00000100:00100000:3.0:1787826310.654538:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346203904:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826311.148310:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346204032:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826311.148404:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826311.148656:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346204032 00000100:00100000:1.0:1787826311.148665:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346204032 00000100:00100000:1.0:1787826311.148723:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6224 00000100:00100000:1.0:1787826311.148733:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346204032:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826311.148759:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662430208 00000100:00100000:1.0:1787826311.148810:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346204032:12345-0@lo:13:osp-pre-0-1.0 Request processed in 78us (406us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826311.148817:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6224 00000100:00100000:1.0:1787826311.148863:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346204032:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826311.149074:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346204288:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826311.149131:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826311.149194:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826311.149201:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346204160:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826311.149233:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826311.149446:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346204160 00000100:00100000:0.0:1787826311.149451:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346204160 00000100:00100000:0.0:1787826311.149500:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23628 00000100:00100000:0.0:1787826311.149509:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf88050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346204160:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787826311.149539:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442799616 avail 1296560128 00000100:00100000:0.0:1787826311.149585:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf88050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346204160:12345-0@lo:41:osp-pre-0-1.0 Request processed in 77us (351us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826311.149591:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23628 00000100:00100000:1.0:1787826311.149969:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346204160:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826311.150299:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346204416:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826311.150342:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826311.150609:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826311.150617:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346204544:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826311.150692:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826311.150790:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346204544 00000100:00100000:1.0:1787826311.150798:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346204544 00000100:00100000:1.0:1787826311.150853:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23629 00000100:00100000:1.0:1787826311.150864:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346204544:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826311.150891:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346204288 00000004:00000024:1.0:1787826311.150892:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443405824 avail 1297166336 00000100:00100000:2.0:1787826311.150899:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346204288 00000100:00100000:1.0:1787826311.150946:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346204544:12345-0@lo:41:osp-pre-1-0.0 Request processed in 84us (252us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826311.150956:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23629 00000100:00100000:2.0:1787826311.150983:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346204416 00000100:00100000:2.0:1787826311.150987:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346204416 00000100:00100000:2.0:1787826311.151004:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6225 00000100:00100000:2.0:1787826311.151010:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346204288:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826311.151024:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683135488 00000100:00100000:2.0:1787826311.151072:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346204288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 62us (1967us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826311.151079:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6225 00000100:00100000:2.0:1787826311.151092:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6226 00000100:00100000:2.0:1787826311.151098:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346204416:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826311.151127:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662430208 00000100:00100000:2.0:1787826311.151155:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346204416:12345-0@lo:13:osp-pre-0-0.0 Request processed in 58us (815us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826311.151161:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6226 00000100:00080000:3.0:1787826311.151411:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826311.151459:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346204544:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826311.151484:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346204416:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826311.151489:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346204288:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1787826311.349638:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826312.365394:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826313.148154:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826313.148503:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332588288 00000100:00100000:0.0:1787826313.148517:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332588288 00000100:00100000:0.0:1787826313.148591:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68405 00000100:00100000:0.0:1787826313.148602:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224895:x1874663332588288:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826313.148621:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826313.148688:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a912 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801417 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.148750:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.148763:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.148776:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826313.148794:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00200000:0.0:1787826313.148824:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826313.148829:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826313.148852:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.148878:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4a912 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.148893:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.148902:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826313.148915:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826313.148918:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826313.148984:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224895:x1874663332588288:12345-192.168.203.56@tcp:101:cat.0 Request processed in 383us (886us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826313.148990:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68405 00000100:00100000:0.0:1787826313.164392:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826313.164468:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332588416 00000100:00100000:1.0:1787826313.164479:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332588416 00000100:00100000:1.0:1787826313.164535:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68406 00000100:00100000:1.0:1787826313.164545:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224895:x1874663332588416:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826313.164565:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826313.164586:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a920 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080141e expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164663:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164677:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164687:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164702:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164709:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164715:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164798:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a935 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164807:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a935 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164813:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a935 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826313.164836:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a935 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164853:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164859:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164865:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164871:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164877:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826313.164881:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164888:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164900:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826313.164914:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826313.164919:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164925:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a927 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164947:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164952:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164957:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164961:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826313.164963:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164968:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164974:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1d00) 00010000:00010000:1.0:1787826313.164977:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826313.164979:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.164984:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a92e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.165015:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a920 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080141e expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.165030:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a935 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080141e expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.165039:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:1.0:1787826313.165041:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826313.165043:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826313.165138:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224895:x1874663332588416:12345-192.168.203.56@tcp:101:cat.0 Request processed in 594us (755us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.165146:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68406 00000100:00100000:3.0:1787826313.168980:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826313.169158:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332588544 00000100:00100000:1.0:1787826313.169169:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332588544 00000100:00100000:1.0:1787826313.169237:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68407 00000100:00100000:1.0:1787826313.169250:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224895:x1874663332588544:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826313.169274:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826313.169296:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a93c lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801425 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.169360:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a943 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.169380:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a943 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.169393:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a943 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826313.169516:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4c:0x0]: returning size 4096 00000004:00200000:1.0:1787826313.169522:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826313.169561:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4a93c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801425 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.169586:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a943 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801425 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.169594:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:1.0:1787826313.169597:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826313.169601:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826313.169667:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224895:x1874663332588544:12345-192.168.203.56@tcp:101:cat.0 Request processed in 420us (698us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.169675:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68407 00000100:00080000:3.0:1787826313.169876:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826313.174739:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826313.177150:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332588672 00000100:00100000:2.0:1787826313.177164:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332588672 00000100:00100000:2.0:1787826313.177234:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68408 00000100:00100000:2.0:1787826313.177244:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224895:x1874663332588672:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826313.177265:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826313.177288:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a94a lrc: 2/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080142c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.177351:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.177365:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.177376:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826313.177420:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x194:0x0]: returning size 4096 00000004:00200000:2.0:1787826313.177425:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826313.177458:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4a94a lrc: 0/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.177476:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.177491:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:2.0:1787826313.177494:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826313.177498:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826313.177563:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224895:x1874663332588672:12345-192.168.203.56@tcp:101:cat.0 Request processed in 320us (2835us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826313.177570:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68408 00000100:00100000:0.0:1787826313.182962:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826313.183087:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332588800 00000100:00100000:1.0:1787826313.183097:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332588800 00000100:00100000:1.0:1787826313.183181:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68409 00000100:00100000:1.0:1787826313.183193:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224895:x1874663332588800:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826313.183214:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826313.183239:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a958 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801433 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.183307:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.183321:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.183332:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826313.183385:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x192:0x0]: returning size 4096 00000004:00200000:1.0:1787826313.183391:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826313.183428:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4a958 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.183446:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.183454:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7845cb1d00) 00010000:00010000:1.0:1787826313.183457:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826313.183460:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826313.183520:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224895:x1874663332588800:12345-192.168.203.56@tcp:101:cat.0 Request processed in 330us (567us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.183526:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68409 00000100:00100000:0.0:1787826313.186971:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826313.187223:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332589056 00000100:00100000:1.0:1787826313.187234:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332589056 00000100:00100000:1.0:1787826313.187297:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68410 00000100:00100000:1.0:1787826313.187308:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224895:x1874663332589056:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826313.187329:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826313.187352:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a966 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080143a expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.187433:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.187447:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.187458:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.187472:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.187480:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.187486:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:1.0:1787826313.187558:0:6527:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00010000:00010000:0.0:1787826313.388487:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826313.669741:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826313.669844:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332589184 00000100:00100000:3.0:1787826313.669857:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332589184 00000100:00100000:3.0:1787826313.669958:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68411 00000100:00100000:3.0:1787826313.669972:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224897:x1874663332589184:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826313.670005:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826313.670031:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a97b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801441 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.670174:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.670190:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.670204:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826313.670265:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826313.670269:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000100:00080000:2.0:1787826313.670285:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00010000:3.0:1787826313.670294:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.670322:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a97b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.670345:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.670355:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826313.670358:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826313.670362:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826313.670430:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224897:x1874663332589184:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 460us (702us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.670442:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68411 00000100:00100000:0.0:1787826313.680753:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826313.680915:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332589312 00000100:00100000:2.0:1787826313.680928:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332589312 00000100:00100000:2.0:1787826313.681001:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68412 00000100:00100000:2.0:1787826313.681015:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224897:x1874663332589312:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826313.681052:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826313.681079:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4a989 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801448 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681194:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681211:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681225:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681243:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681251:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681260:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681346:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681356:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681364:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826313.681388:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681403:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681411:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681419:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681427:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681435:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.681439:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681449:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681464:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826313.681468:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.681472:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681480:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a990 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681496:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681504:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681510:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681515:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.681518:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681525:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681533:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cdb00) 00010000:00010000:2.0:1787826313.681535:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.681538:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681544:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a997 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681584:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4a989 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681598:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.681607:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:2.0:1787826313.681609:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826313.681612:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826313.681706:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224897:x1874663332589312:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 693us (967us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826313.681715:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68412 00000100:00100000:1.0:1787826313.687228:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826313.687388:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332589440 00000100:00100000:0.0:1787826313.687401:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332589440 00000100:00100000:0.0:1787826313.687491:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68413 00000100:00100000:0.0:1787826313.687504:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224897:x1874663332589440:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826313.687538:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826313.687564:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4a9a5 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080144f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.687652:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4a9ac lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.687672:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4a9ac lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.687687:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4a9ac lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826313.687809:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4c:0x0]: returning size 4096 00000004:00200000:0.0:1787826313.687815:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826313.687857:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4a9a5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080144f expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.687880:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4a9ac lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080144f expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.687897:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:0.0:1787826313.687900:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826313.687905:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826313.687981:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224897:x1874663332589440:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 479us (767us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826313.687989:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68413 00000100:00080000:2.0:1787826313.688188:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826313.697822:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826313.697932:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332589568 00000100:00100000:3.0:1787826313.697945:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332589568 00000100:00100000:3.0:1787826313.698016:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68414 00000100:00100000:3.0:1787826313.698029:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224897:x1874663332589568:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826313.698054:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826313.698079:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9b3 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801456 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698212:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 3/1,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698230:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698244:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698256:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698262:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698269:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698370:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a9c8 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698383:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a9c8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698390:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a9c8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826313.698436:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4d:0x0]: returning size 6 00000004:00200000:3.0:1787826313.698441:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826313.698462:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a9c8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698476:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a9c8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698486:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698493:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698499:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698506:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698512:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826313.698517:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698525:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698541:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:3.0:1787826313.698544:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826313.698550:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 3/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698557:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4a9ba lrc: 0/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698573:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698579:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698585:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698590:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826313.698592:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698597:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698604:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x4ec94fd6 (ffff9f7952ca9300) 00010000:00010000:3.0:1787826313.698606:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826313.698609:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698615:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4a9c1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698657:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9b3 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801456 expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698672:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a9c8 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801456 expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.698680:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f79460a2300) 00010000:00010000:3.0:1787826313.698683:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826313.698685:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826313.698753:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224897:x1874663332589568:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 727us (944us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.698761:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68414 00000100:00100000:3.0:1787826313.701589:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826313.701711:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332589696 00000100:00100000:2.0:1787826313.701724:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332589696 00000100:00100000:2.0:1787826313.701802:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68415 00000100:00100000:2.0:1787826313.701813:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224897:x1874663332589696:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826313.701841:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826313.701867:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4a9cf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080145d expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.701949:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.701966:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.701985:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.701997:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702005:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702012:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702329:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a9e4 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702339:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a9e4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702347:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a9e4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826313.702395:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dba:0x0]: returning size 6 00000004:00200000:2.0:1787826313.702400:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826313.702422:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a9e4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702436:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a9e4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702446:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702457:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702464:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702470:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702477:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.702481:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702489:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702503:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7845cb1d00) 00010000:00010000:2.0:1787826313.702506:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.702512:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702519:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4a9d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702536:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702543:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702549:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702555:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.702558:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702564:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702571:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f79460a2b00) 00010000:00010000:2.0:1787826313.702574:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.702577:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702583:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4a9dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702610:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4a9cf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080145d expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702624:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a9e4 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080145d expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.702641:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79785cdb00) 00010000:00010000:2.0:1787826313.702644:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826313.702647:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826313.702727:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224897:x1874663332589696:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 915us (1153us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826313.702738:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68415 00000100:00100000:1.0:1787826313.717727:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826313.717875:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826313.717950:0:6540:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332590080 00000100:00100000:2.0:1787826313.717968:0:6540:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332590080 00000100:00100000:2.0:1787826313.718084:0:6540:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1229 00000100:00100000:2.0:1787826313.718100:0:6540:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224897:x1874663332590080:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:2.0:1787826313.718205:0:6540:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826313.718239:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4a9e4 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080145d expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.718267:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79785cdb00) 00010000:00010000:2.0:1787826313.718285:0:6540:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826313.718297:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332589824 00000100:00100000:1.0:1787826313.718308:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332589824 00000100:00100000:1.0:1787826313.718386:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11085 00000100:00100000:2.0:1787826313.718391:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fd500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346204672:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.718403:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:2393:x1874663332589824:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:1.0:1787826313.718422:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826313.718449:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4a9c8 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801456 expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.718469:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f79460a2300) 00010000:00010000:1.0:1787826313.718480:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826313.718535:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2393:x1874663332589824:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 138us (660us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.718542:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11085 00000100:00100000:2.0:1787826313.718676:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826313.718708:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346204672 00000100:00100000:1.0:1787826313.718716:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346204672 00000100:00100000:2.0:1787826313.718762:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826313.718769:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23630 00000100:00100000:1.0:1787826313.718784:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346204672:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.718877:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346204672:12345-0@lo:1000:mdt_io00_000.0 Request processed in 96us (203us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.718884:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23630 00000100:00100000:2.0:1787826313.718947:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fd500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346204672:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.719292:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fd500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346204800:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.719339:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826313.719459:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.721758:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346204800 00000100:00100000:3.0:1787826313.721777:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346204800 00000100:00100000:3.0:1787826313.721863:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23631 00000100:00100000:3.0:1787826313.721877:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346204800:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.724761:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346204800:12345-0@lo:1000:mdt_io00_000.0 Request processed in 2873us (5409us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.724779:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23631 00000100:00100000:2.0:1787826313.724785:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fd500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346204800:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826313.724850:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x194:0x0] 00000100:00100000:2.0:1787826313.724919:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fd500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346204928:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.724966:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826313.725020:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.725557:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346204928 00000100:00100000:3.0:1787826313.725567:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346204928 00000100:00100000:3.0:1787826313.725874:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23632 00000100:00100000:3.0:1787826313.725906:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346204928:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.726593:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346204928:12345-0@lo:1000:mdt_io00_000.0 Request processed in 695us (1624us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.726606:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23632 00000100:00100000:0.0:1787826313.726724:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fd500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346204928:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826313.726779:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4c:0x0] 00000100:00100000:0.0:1787826313.726844:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346205056:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.726893:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.730548:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826313.732408:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346205056 00000100:00100000:2.0:1787826313.732428:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346205056 00000100:00100000:1.0:1787826313.732856:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23633 00000100:00100000:1.0:1787826313.732881:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346205056:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.733449:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346205056:12345-0@lo:1000:mdt_io00_000.0 Request processed in 578us (6554us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.733459:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23633 00000100:00100000:0.0:1787826313.733688:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9df800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346205056:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826313.733791:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x194:0x0] 00000100:00100000:0.0:1787826313.733855:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346205184:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.733919:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826313.740255:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346205184 00000100:00100000:3.0:1787826313.740278:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346205184 00000100:00100000:3.0:1787826313.740373:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23634 00000100:00100000:3.0:1787826313.740387:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346205184:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.740734:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346205184:12345-0@lo:1000:mdt_io00_000.0 Request processed in 348us (6813us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.740744:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23634 00000100:00100000:0.0:1787826313.742194:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826313.742253:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346205184:0@lo:1000:mdt_io00_000.0 00000004:00020000:0.0:1787826313.742337:0:6540:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:0.0:1787826313.742411:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 3/0,1 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.742424:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 3/0,1 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.742799:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 3/0,1 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826313.742826:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346205312:0@lo:101:mdt_io00_000.0 00000100:00100000:0.0:1787826313.742897:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00020000:3.0:1787826313.746357:0:6527:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4440 00010000:00010000:3.0:1787826313.746518:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746544:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746561:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826313.746578:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dba:0x0]: returning size 6 00010000:00010000:3.0:1787826313.746715:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746724:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746732:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746745:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746758:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826313.746764:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746773:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746789:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7845cb1d00) 00010000:00010000:3.0:1787826313.746792:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826313.746796:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 3/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746804:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4a96d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746823:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746830:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746837:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746842:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826313.746845:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746851:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746863:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826313.746866:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826313.746869:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746876:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4a974 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746931:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4a966 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746947:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.746954:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:3.0:1787826313.746956:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826313.746959:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826313.747075:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224895:x1874663332589056:12345-192.168.203.56@tcp:101:cat.0 Request processed in 559766us (560110us total) trans 8589957384 rc 0/0 00000100:00100000:3.0:1787826313.747086:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68410 00000100:00100000:3.0:1787826313.747353:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346205312 00000100:00100000:3.0:1787826313.747364:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346205312 00000100:00100000:3.0:1787826313.747420:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23635 00000100:00100000:3.0:1787826313.747492:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346205312:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787826313.747517:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826313.747538:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4a9eb expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.747569:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 2/0,1 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.747580:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.747591:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 5/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a9eb expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.747649:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.747671:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 8 pid: 35094 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826313.748215:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.748828:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346205440:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826313.748965:0:6532:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:00080000:3.0:1787826313.749014:0:6532:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00000100:00100000:3.0:1787826313.749192:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81240 going to sleep for 11 seconds 00000001:00080000:1.0:1787826313.753298:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985393 is committed 00010000:00010000:2.0:1787826313.753325:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787826313.753332:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784f7d7a80 refcount 2 committed 0 result 0 batchid 3413 00000020:00080000:1.0:1787826313.753346:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f7d7000 00000020:00080000:1.0:1787826313.753361:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1516 00010000:00010000:2.0:1787826313.753363:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787826313.753366:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a191400 00000020:00080000:1.0:1787826313.753386:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1198 00010000:00010000:2.0:1787826313.753389:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787826313.753404:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3411 batchid updated 00000001:00080000:1.0:1787826313.753413:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985390 is committed 00000020:00080000:1.0:1787826313.753419:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784a8bf840 refcount 2 committed 0 result 0 batchid 3412 00000020:00080000:1.0:1787826313.753423:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc2c100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a8bfb40 00000020:00080000:1.0:1787826313.753427:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1515 00000020:00080000:1.0:1787826313.753429:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7946083600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852be1600 00000020:00080000:1.0:1787826313.753433:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1197 00000001:00080000:1.0:1787826313.753445:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985388 is committed 00000020:00080000:3.0:1787826313.753457:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3413 committed batchid 3411 00000004:00080000:3.0:1787826313.753645:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81180 with version 6828 00000020:00080000:3.0:1787826313.753663:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3410 cancel update log [0x200000401:0x3:0x0].1513: rc = 0 00000020:00080000:3.0:1787826313.753714:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3410 cancel update log [0x240000400:0x3:0x0].1195: rc = 0 00000004:00080000:3.0:1787826313.753744:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81780 with version 6829 00000020:00080000:3.0:1787826313.753749:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3411 cancel update log [0x200000401:0x3:0x0].1514: rc = 0 00000020:00080000:3.0:1787826313.753767:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3411 cancel update log [0x240000400:0x3:0x0].1196: rc = 0 00000020:00080000:3.0:1787826313.753773:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3411 00000004:00080000:0.0:1787826313.753897:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6828 rpc_version 6828 00000020:00100000:0.0:1787826313.753943:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826313.753958:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f785124f480 x1874663346205696/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826313.753987:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346205696:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826313.754088:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.754161:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7cc0 going to sleep for 11 seconds 00010000:00010000:2.0:1787826313.754519:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.754527:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754539:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754560:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:2.0:1787826313.754564:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.754570:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754578:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4a84e lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754596:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754603:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754611:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754617:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.754621:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754643:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754652:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:2.0:1787826313.754654:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.754661:0:3639:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a9eb expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754674:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754681:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4a855 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754702:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754709:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754715:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754721:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.754724:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754731:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754744:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x4ec94fd6 (ffff9f7952ca9300) 00010000:00010000:2.0:1787826313.754747:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.754750:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.754757:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4a8f6 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.754835:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346205696 00000100:00100000:3.0:1787826313.754846:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346205696 00000100:00100000:3.0:1787826313.754910:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23636 00000100:00100000:3.0:1787826313.754922:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346205696:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826313.755593:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346205696:12345-0@lo:1000:osp_up0-1.0 Request processed in 669us (1501us total) trans 8589957385 rc 0/0 00000100:00100000:3.0:1787826313.755604:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23636 00000100:00100000:3.0:1787826313.759807:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826313.759952:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826313.759998:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332590336 00000100:00100000:3.0:1787826313.760010:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332590336 00010000:00010000:3.0:1787826313.760061:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 7 pid: 35094 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.760102:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21937 00000100:00100000:3.0:1787826313.760145:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332590336:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826313.760163:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826313.760171:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 7 pid: 35094 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.760187:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 7 pid: 35094 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.760214:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:3.0:1787826313.760224:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4a9eb expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.760237:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a9eb expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.760247:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4a9eb expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.760297:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a9eb expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.760314:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6513:1874663346205824:0@lo:105: 00000100:00100000:3.0:1787826313.760377:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826313.760422:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.760456:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826313.760469:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346205824 00000100:00100000:3.0:1787826313.760478:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346205824 00000100:00100000:3.0:1787826313.760513:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7860 00000100:00100000:2.0:1787826313.760518:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332590208 00000100:00100000:3.0:1787826313.760523:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78423fb850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6513:x1874663346205824:12345-0@lo:105: 00000100:00100000:2.0:1787826313.760526:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332590208 00010000:00010000:3.0:1787826313.760540:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 4/0,1 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.760552:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 4/0,1 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.760560:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 4/0,1 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.760573:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826313.760579:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30824 00010000:00010000:3.0:1787826313.760586:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826313.760594:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332590208:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826313.760598:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.760616:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826313.760617:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43800) 00010000:00010000:2.0:1787826313.760647:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4aa07 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801464 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.760669:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78423fb850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6513:x1874663346205824:12345-0@lo:105: Request processed in 147us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.760677:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7860 00000100:00100000:3.0:1787826313.760760:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6513:1874663346205824:0@lo:105: 00010000:00010000:3.0:1787826313.760779:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826313.760820:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332590336:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 679us (869us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.760829:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21937 00000100:00100000:2.0:1787826313.760834:0:8445:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346205952:192.168.203.56@tcp:106: 00000100:00080000:3.0:1787826313.760866:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826313.760890:0:8445:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00010000:00010000:1.0:1787826313.761151:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4a951 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080142c expref: 6 pid: 35094 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826313.761172:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346205440:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826313.761197:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4a9eb expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.761207:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:1.0:1787826313.761211:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826313.761214:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:1.0:1787826313.761268:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346205312:12345-0@lo:101:mdt_io00_000.0 Request processed in 13782us (18370us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.761275:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23635 00000100:00100000:2.0:1787826313.761613:0:6540:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985388 gen 2 00000004:00080000:2.0:1787826313.761622:0:6540:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985388 committed transno 4294985388 00000020:00080000:2.0:1787826313.761641:0:6540:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f784549f600 refcount 2 committed 0 result 0 batchid 4294979081 00000020:00080000:2.0:1787826313.761647:0:6540:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784463b400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784549fd80 00000020:00080000:2.0:1787826313.761651:0:6540:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].937 00000020:00080000:2.0:1787826313.761654:0:6540:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784463b600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7842367c00 00000020:00080000:2.0:1787826313.761656:0:6540:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].934 00000100:00100000:2.0:1787826313.761802:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346205312:0@lo:101:mdt_io00_000.0 00000020:00080000:3.0:1787826313.761812:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979081 committed batchid 4294979080 00010000:00010000:2.0:1787826313.761823:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826313.761848:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x194:0x0] 00010000:00010000:2.0:1787826313.761893:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.761901:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.761910:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826313.761929:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346206080:0@lo:101:mdt_io00_000.0 00000100:00100000:2.0:1787826313.761967:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826313.762001:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826313.762032:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346206080 00000100:00100000:1.0:1787826313.762039:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346206080 00000100:00100000:1.0:1787826313.762088:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23637 00000100:00100000:1.0:1787826313.762098:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346206080:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787826313.762133:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826313.762151:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aa15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4aa0e expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.762176:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aa15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aa0e expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.762190:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aa15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aa0e expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.762200:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x4ec94fd6 (ffff9f794385ad00) 00010000:00010000:1.0:1787826313.762204:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826313.762207:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826313.762240:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346206080:12345-0@lo:101:mdt_io00_000.0 Request processed in 143us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.762247:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23637 00000020:00080000:3.0:1787826313.762403:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979080 00000100:00100000:2.0:1787826313.762603:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346206080:0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826313.762621:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aa15 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.762641:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aa15 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826313.762660:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x194:0x0] 00010000:00010000:2.0:1787826313.762764:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.762779:0:6540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.762793:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 4/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.762824:0:6540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.762840:0:6540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 10 pid: 24703 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826313.762854:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346206208:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826313.762931:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153780 going to sleep for 11 seconds 00002000:00010000:2.0:1787826313.763923:0:8445:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000401:0x51f:0x0] updating lvb size: 0 -> 6 00002000:00010000:2.0:1787826313.764018:0:8445:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000401:0x51f:0x0] updating lvb mtime: 0 -> 1787826308 00002000:00010000:2.0:1787826313.764022:0:8445:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000401:0x51f:0x0] updating lvb atime: 0 -> 1787826308 00002000:00010000:2.0:1787826313.764024:0:8445:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000401:0x51f:0x0] updating lvb ctime: 0 -> 1787826308 00002000:00010000:2.0:1787826313.764027:0:8445:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000401:0x51f:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:2.0:1787826313.764052:0:8445:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346205952:192.168.203.56@tcp:106: 00010000:00010000:2.0:1787826313.764091:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4aa07 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801464 expref: 6 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.764253:0:8445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4aa07 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801464 expref: 6 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.764270:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826313.764596:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332590208:12345-192.168.203.56@tcp:101:cat.0 Request processed in 4003us (4802us total) trans 0 rc 301/301 00000100:00100000:2.0:1787826313.764609:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30824 00000100:00100000:0.0:1787826313.764680:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346205696:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826313.764696:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6829 rpc_version 6829 00000020:00100000:0.0:1787826313.764728:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826313.764737:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f797c320000 x1874663346206464/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826313.764760:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346206464:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826313.764811:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.764847:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.765193:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346206464 00000100:00100000:3.0:1787826313.765204:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346206464 00000100:00100000:3.0:1787826313.765262:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23638 00000100:00100000:3.0:1787826313.765274:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346206464:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826313.765467:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346206464:12345-0@lo:1000:osp_up0-1.0 Request processed in 196us (656us total) trans 8589957386 rc 0/0 00000100:00100000:3.0:1787826313.765477:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23638 00000100:00100000:3.0:1787826313.766201:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826313.766245:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346206208:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826313.766280:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826313.766810:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332590464 00000100:00100000:1.0:1787826313.766821:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332590464 00010000:00010000:1.0:1787826313.766866:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 11 pid: 24703 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826313.766916:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21938 00000100:00100000:1.0:1787826313.766928:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332590464:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826313.766956:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826313.766964:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 11 pid: 24703 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.766979:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 11 pid: 24703 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.767003:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4a95f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801433 expref: 11 pid: 24703 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.767026:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7845cb1d00) 00010000:00010000:1.0:1787826313.767032:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.767043:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.767057:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.767083:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826313.767177:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332590464:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 250us (983us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.767187:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21938 00000100:00100000:0.0:1787826313.770331:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346206464:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826313.770795:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.770808:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.770845:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.770861:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.770873:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826313.770971:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346206592:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.771054:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.771232:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826313.771678:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826313.771752:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332590592 00000100:00100000:2.0:1787826313.771762:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332590592 00000100:00100000:2.0:1787826313.771835:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30825 00000100:00100000:2.0:1787826313.771858:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:224895:x1874663332590592:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826313.771879:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826313.771898:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080146b expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.771935:0:35096:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.771945:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x51f:0x0].0x0 since=4s v=(0 0) 00010000:00010000:2.0:1787826313.771949:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826313.771955:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 4/0,0 mode: --/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.771987:0:35096:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.772003:0:35096:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 5 pid: 8445 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826313.772022:0:35096:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:35096:1874663346206720:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826313.772156:0:35096:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826313.773430:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346206592 00000100:00100000:1.0:1787826313.773442:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346206592 00000100:00100000:1.0:1787826313.773512:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23639 00000100:00100000:1.0:1787826313.773524:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346206592:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.773671:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346206592:12345-0@lo:1000:mdt_io00_000.0 Request processed in 149us (2617us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.773679:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23639 00000100:00100000:0.0:1787826313.773766:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346206592:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.773852:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346206848:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.773890:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.773927:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.774029:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346206848 00000100:00100000:3.0:1787826313.774034:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346206848 00000100:00100000:3.0:1787826313.774073:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23640 00000100:00100000:3.0:1787826313.774082:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346206848:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.774761:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346206848:12345-0@lo:1000:mdt_io00_000.0 Request processed in 678us (868us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.774770:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23640 00000100:00100000:0.0:1787826313.774951:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346206848:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.775015:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346206976:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.775044:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.775068:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.775317:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346206976 00000100:00100000:3.0:1787826313.775324:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346206976 00000100:00100000:2.0:1787826313.775373:0:35096:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:35096:1874663346206720:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826313.775394:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23641 00010000:00010000:2.0:1787826313.775404:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 3/0,0 mode: --/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.775404:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346206976:12345-0@lo:1000:mdt_io00_000.0 00010000:00010000:2.0:1787826313.775429:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826313.775482:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224895:x1874663332590592:12345-192.168.203.56@tcp:101:cat.0 Request processed in 3627us (3807us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826313.775490:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30825 00000100:00100000:3.0:1787826313.775516:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346206976:12345-0@lo:1000:mdt_io00_000.0 Request processed in 113us (471us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826313.775522:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23641 00000100:00100000:0.0:1787826313.775694:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346206976:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.775768:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207104:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.775793:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.775821:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.776086:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346207104 00000100:00100000:3.0:1787826313.776092:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346207104 00000100:00100000:3.0:1787826313.776157:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23642 00000100:00100000:3.0:1787826313.776166:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346207104:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.776236:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346207104:12345-0@lo:1000:mdt_io00_000.0 Request processed in 71us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.776244:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23642 00000100:00100000:0.0:1787826313.776381:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207104:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.776460:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207232:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.776487:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.776513:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826313.776529:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346207232 00000100:00100000:1.0:1787826313.776535:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346207232 00000100:00100000:1.0:1787826313.776605:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23643 00000100:00100000:1.0:1787826313.776614:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346207232:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.776714:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346207232:12345-0@lo:1000:mdt_io00_000.0 Request processed in 101us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.776721:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23643 00000100:00100000:0.0:1787826313.776778:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207232:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826313.776799:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4c:0x0] 00010000:00010000:0.0:1787826313.776896:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.776922:0:6540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.776935:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.777028:0:6540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.777046:0:6540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 9 pid: 6527 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826313.777061:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346207360:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826313.777219:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826313.779416:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826313.779460:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332590720 00000100:00100000:1.0:1787826313.779472:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332590720 00010000:00010000:1.0:1787826313.779542:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 6 pid: 8445 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826313.779626:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21939 00000100:00100000:1.0:1787826313.779649:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332590720:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826313.779669:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826313.779767:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 6 pid: 8445 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.779782:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 6 pid: 8445 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.779811:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e826c654/0xe21309aa81f4a8e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801402 expref: 6 pid: 8445 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.779834:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826313.779839:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826313.779848:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.779859:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.779905:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826313.779920:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:125406:1874663346207488:192.168.203.56@tcp:105: 00000100:00100000:1.0:1787826313.780016:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78555e70c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826313.780161:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346207360:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826313.780189:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826313.780619:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826313.781128:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826313.781263:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332590848 00000100:00100000:3.0:1787826313.781274:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332590848 00010000:00010000:3.0:1787826313.781613:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 10 pid: 6527 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.781680:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21940 00000100:00100000:3.0:1787826313.781770:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971594050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2396:x1874663332590848:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826313.781922:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826313.781931:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 10 pid: 6527 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.781946:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 10 pid: 6527 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.781975:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4a9f2 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dba:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080143a expref: 10 pid: 6527 timeout: 8225 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.782000:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:3.0:1787826313.782007:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.782019:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.782028:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.782067:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826313.782162:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971594050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332590848:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 395us (1055us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.782171:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21940 00010000:00010000:0.0:1787826313.782452:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.782463:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.782506:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:125406:1874663346207488:192.168.203.56@tcp:105: 00010000:00010000:3.0:1787826313.782531:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826313.782539:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.782548:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.782553:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332590720:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2908us (3148us total) trans 0 rc 0/0 00010000:00010000:0.0:1787826313.782559:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826313.782560:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21939 00000100:00100000:0.0:1787826313.782581:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207616:0@lo:101:mdt_io00_000.0 00000100:00100000:0.0:1787826313.782629:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.782676:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.782842:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346207616 00000100:00100000:3.0:1787826313.782850:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346207616 00000100:00100000:3.0:1787826313.782967:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23644 00000100:00100000:3.0:1787826313.782977:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346207616:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787826313.782997:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826313.783018:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4aa3f lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4aa38 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.783041:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4aa3f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aa38 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.783055:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4aa3f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aa38 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826313.783063:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826313.783066:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1787826313.783069:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826313.783151:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346207616:12345-0@lo:101:mdt_io00_000.0 Request processed in 176us (521us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.783158:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23644 00000100:00100000:0.0:1787826313.783466:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207616:0@lo:101:mdt_io00_000.0 00010000:00010000:0.0:1787826313.783484:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aa3f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.783497:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aa3f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826313.783517:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4d:0x0] 00000100:00100000:0.0:1787826313.783576:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207744:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.783615:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826313.783629:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346207744 00000100:00100000:0.0:1787826313.783643:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826313.783645:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346207744 00000100:00100000:1.0:1787826313.783700:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23645 00000100:00100000:1.0:1787826313.783709:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346207744:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.783806:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346207744:12345-0@lo:1000:mdt_io00_000.0 Request processed in 98us (190us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.783814:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23645 00000100:00100000:0.0:1787826313.783963:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207744:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826313.783990:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x194:0x0] 00000100:00100000:0.0:1787826313.784019:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207872:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.784096:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826313.784142:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826313.784189:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826313.784263:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332590976 00000100:00100000:2.0:1787826313.784269:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332590976 00000100:00100000:2.0:1787826313.784317:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68416 00000100:00100000:2.0:1787826313.784326:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224895:x1874663332590976:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826313.784340:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826313.784360:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4aa46 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801472 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.784421:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4aa46 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801472 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.784436:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4aa46 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801472 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.784462:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:2.0:1787826313.784466:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787826313.784469:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:3.0:1787826313.784485:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346207872 00000100:00100000:3.0:1787826313.784489:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346207872 00000100:00100000:2.0:1787826313.784516:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224895:x1874663332590976:12345-192.168.203.56@tcp:101:cat.0 Request processed in 191us (409us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826313.784523:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68416 00000100:00100000:3.0:1787826313.784549:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23646 00000100:00100000:3.0:1787826313.784556:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346207872:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.784625:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346207872:12345-0@lo:1000:mdt_io00_000.0 Request processed in 69us (528us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.784644:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23646 00000100:00100000:0.0:1787826313.784857:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346207872:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826313.784883:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4c:0x0] 00010000:00010000:0.0:1787826313.784960:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.784971:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.784980:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826313.785037:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208000:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.785067:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826313.785159:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826313.785297:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346208000 00000100:00100000:1.0:1787826313.785303:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346208000 00000100:00100000:0.0:1787826313.785498:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23647 00000100:00100000:0.0:1787826313.785508:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208000:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.785599:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208000:12345-0@lo:1000:mdt_io00_000.0 Request processed in 91us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826313.785606:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23647 00000100:00100000:1.0:1787826313.785671:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208000:0@lo:1000:mdt_io00_000.0 00000004:00080000:1.0:1787826313.785694:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4c:0x0] 00000100:00100000:1.0:1787826313.785726:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208128:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.785753:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826313.785858:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346208128 00000100:00100000:2.0:1787826313.785864:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346208128 00000100:00100000:1.0:1787826313.785871:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78555e73c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826313.785891:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23648 00000100:00100000:2.0:1787826313.785899:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208128:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.785968:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208128:12345-0@lo:1000:mdt_io00_000.0 Request processed in 69us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826313.785975:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23648 00000100:00100000:1.0:1787826313.786026:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208128:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.786064:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208256:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.786087:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826313.786136:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78555e73c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826313.786189:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346208256 00000100:00100000:3.0:1787826313.786193:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346208256 00000100:00100000:3.0:1787826313.786225:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23649 00000100:00100000:3.0:1787826313.786233:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208256:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.786303:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208256:12345-0@lo:1000:mdt_io00_000.0 Request processed in 70us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826313.786309:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23649 00000100:00100000:2.0:1787826313.786418:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208256:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826313.786440:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x194:0x0] 00000100:00100000:2.0:1787826313.786469:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208384:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.786499:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826313.786553:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826313.786614:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346208384 00000100:00100000:0.0:1787826313.786620:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346208384 00000100:00100000:0.0:1787826313.786658:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23650 00000100:00100000:0.0:1787826313.786665:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208384:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826313.786788:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208384:12345-0@lo:1000:mdt_io00_000.0 Request processed in 119us (284us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826313.786795:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23650 00000100:00100000:2.0:1787826313.786814:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208384:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826313.786832:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4c:0x0] 00000100:00100000:2.0:1787826313.786859:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208512:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.786882:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826313.786947:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826313.787036:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346208512 00000100:00100000:1.0:1787826313.787041:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346208512 00000100:00100000:1.0:1787826313.787077:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23651 00000100:00100000:1.0:1787826313.787084:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208512:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826313.787881:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826313.788014:0:24159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332591104 00000100:00100000:2.0:1787826313.788022:0:24159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332591104 00002000:00100000:2.0:1787826313.788071:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000401:0x51f:0x0] (0->4095) req@ffff9f796cdb1180 x1874663332591104/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:669/0 lens 488/0 e 0 to 0 dl 1787826324 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826313.788083:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f796cdb1180 with x1874663332591104 ext(0->4095) 00002000:00010000:2.0:1787826313.788089:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f796cdb1180 00000100:00100000:2.0:1787826313.788147:0:24159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27647 00000100:00100000:2.0:1787826313.788159:0:24159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332591104:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:2.0:1787826313.788532:0:24159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332591104:12345-192.168.203.56@tcp:3:cat.0 Request processed in 373us (656us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787826313.788540:0:24159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27647 00002000:00100000:2.0:1787826313.788548:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x280000401:0x51f:0x0] (0->4095) req@ffff9f796cdb1180 x1874663332591104/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:669/0 lens 488/4536 e 0 to 0 dl 1787826324 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826313.788556:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f796cdb1180 with x1874663332591104 ext(0->4095) 00002000:00010000:2.0:1787826313.788560:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f796cdb1180 00000100:00100000:0.0:1787826313.788707:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208512:12345-0@lo:1000:mdt_io00_000.0 Request processed in 1622us (1822us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826313.788716:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23651 00000100:00100000:3.0:1787826313.788852:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208512:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.788946:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208640:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826313.788987:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826313.789019:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826313.789134:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346208640 00000100:00100000:2.0:1787826313.789141:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346208640 00000100:00100000:2.0:1787826313.789181:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23652 00000100:00100000:2.0:1787826313.789190:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208640:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.789263:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208640:12345-0@lo:1000:mdt_io00_000.0 Request processed in 73us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826313.789270:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23652 00000100:00100000:3.0:1787826313.789353:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208640:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.794082:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208768:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.794227:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826313.794273:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826313.796130:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346208768 00000100:00100000:2.0:1787826313.796151:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346208768 00000100:00100000:2.0:1787826313.797868:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23653 00000100:00100000:2.0:1787826313.798073:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208768:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826313.802235:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346208768:12345-0@lo:1000:mdt_io00_000.0 Request processed in 4335us (7994us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826313.802264:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23653 00000100:00100000:0.0:1787826313.804382:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346208768:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826313.807783:0:6540:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1dba:0x0] is inserted into orphan list, open count = 1 00000040:00080000:2.0:1787826313.807949:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].935, 1240 off879488 00000040:00080000:2.0:1787826313.808033:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].938, 1320 off978632 00000004:00080000:2.0:1787826313.808042:0:6540:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784f7d7e40 with version 1824 00000004:00080000:1.0:1787826313.808158:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1824 rpc_version 1824 00000020:00100000:1.0:1787826313.808239:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826313.808252:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f796cdb3100 x1874663346209024/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826313.808286:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb3100 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346209024:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826313.808364:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826313.808424:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78555e7a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826313.808654:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826313.808755:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332591360 00000100:00100000:0.0:1787826313.808768:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332591360 00000100:00100000:0.0:1787826313.808878:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9340 00000100:00100000:0.0:1787826313.808891:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd2450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224895:x1874663332591360:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:0.0:1787826313.809173:0:6530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1dba:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1787826313.809245:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].213, 64 off21824 00000004:00080000:0.0:1787826313.809270:0:6530:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8126021163317ns, next at 8127021160490ns, rc = 0 00010000:00010000:0.0:1787826313.809344:0:6530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.809365:0:6530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.809381:0:6530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.809390:0:6530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.809401:0:6530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.809407:0:6530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.809423:0:6530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826313.809524:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd2450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224895:x1874663332591360:12345-192.168.203.56@tcp:35:cat.0 Request processed in 634us (875us total) trans 8589957388 rc 0/0 00000100:00100000:0.0:1787826313.809533:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9340 00010000:00010000:1.0:1787826313.809671:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826313.809684:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.809695:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826313.809700:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.809708:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.809727:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:1.0:1787826313.809730:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826313.809734:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.809739:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4aa54 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dba:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.809755:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826313.809757:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826313.809759:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826313.810721:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346209024 00000100:00100000:3.0:1787826313.810736:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346209024 00000100:00100000:3.0:1787826313.810808:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23654 00000100:00100000:3.0:1787826313.810819:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346209024:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826313.815519:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346209024:12345-0@lo:1000:osp_up1-0.0 Request processed in 4689us (7142us total) trans 4294985394 rc 0/0 00000100:00100000:2.0:1787826313.815538:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23654 00000100:00100000:1.0:1787826313.815620:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb3100 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346209024:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787826313.815694:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815708:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815717:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815724:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.815729:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815737:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815763:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:2.0:1787826313.815766:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826313.815772:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815778:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aa31 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826313.815798:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4d:0x0] 00010000:00010000:2.0:1787826313.815805:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aa3f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815814:0:6540:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aa3f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815829:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815835:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815840:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815844:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.815847:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815853:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815864:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f7845cb1d00) 00010000:00010000:2.0:1787826313.815866:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.815869:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815875:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4aa1c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826313.815894:0:6540:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000055d5b341 reply state 00000000c0c460ee transno 8589957387 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815908:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815915:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f79460a2b00) 00010000:00010000:2.0:1787826313.815918:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826313.815920:0:6540:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:2.0:1787826313.815923:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x194:0x0] 00010000:00010000:2.0:1787826313.815927:0:6540:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815934:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815941:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815972:0:6540:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826313.815977:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x194:0x0] 00010000:00010000:2.0:1787826313.815982:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aa15 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815989:0:6540:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aa15 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.815998:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.816004:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.816009:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.816013:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826313.816016:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.816022:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.816028:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f784b4a5d00) 00010000:00010000:2.0:1787826313.816030:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826313.816033:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826313.816038:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4aa4d lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.816237:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826313.816245:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826313.816335:0:6540:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224897:x1874663332590080:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 98235us (98615us total) trans 8589957387 rc 0/0 00000100:00100000:2.0:1787826313.816347:0:6540:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1229 00010000:00010000:0.0:1787826313.816396:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826313.816401:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826313.816409:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.816449:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 2/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.816456:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826313.816495:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 1/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.816502:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4a9eb lrc: 0/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aa00 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826313.816521:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826313.816523:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826313.816525:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826313.816570:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346209152:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826313.816647:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826313.816947:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346209152 00000100:00100000:1.0:1787826313.816956:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346209152 00000100:00100000:1.0:1787826313.817021:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11086 00000100:00100000:1.0:1787826313.817033:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346209152:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826313.817049:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826313.817073:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aa00 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4a9eb expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826313.817090:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:1.0:1787826313.817099:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826313.817216:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346209152:12345-0@lo:103:ldlm_bl.0 Request processed in 185us (585us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826313.817224:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11086 00000100:00100000:0.0:1787826313.817384:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346209152:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826314.352734:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4aa15 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826314.352758:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4aa15 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.352830:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aa15 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.352837:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826314.352867:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee800/0xe21309aa81f4aa0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aa15 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.352914:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4aa3f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826314.352922:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4aa3f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.352941:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aa3f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.352947:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826314.352973:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4aa38 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aa3f expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826314.352988:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957388 is committed 00000004:00080000:0.0:1787826314.352995:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 212 committed 00000001:00080000:0.0:1787826314.353021:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957387 is committed 00000001:00080000:0.0:1787826314.353039:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957386 is committed 00000020:00080000:0.0:1787826314.353046:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979081 batchid updated 00000100:00100000:2.0:1787826314.353138:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787826314.353172:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346209280:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826314.353311:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843091180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346209408:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826314.353417:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826314.353517:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.353530:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.353537:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.353543:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826314.353547:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.353554:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.353566:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826314.353569:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826314.353573:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.353579:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4a8cc lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787826314.353606:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979081 00000004:00080000:3.0:1787826314.353684:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784d9b1540 with version 1825 00000020:00080000:3.0:1787826314.353693:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979081 cancel update log [0x240000404:0x2:0x0].937: rc = 0 00000004:00080000:1.0:1787826314.354268:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1825 rpc_version 1825 00000020:00100000:1.0:1787826314.354286:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826314.354296:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f796cdb2a00 x1874663346209792/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826314.354315:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346209792:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826314.354372:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346209408 00000100:00100000:0.0:1787826314.354381:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346209408 00000020:00080000:3.0:1787826314.354535:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979081 cancel update log [0x200000bd0:0x2:0x0].934: rc = 0 00000020:00080000:3.0:1787826314.354545:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979081 00010000:00010000:3.0:1787826314.354582:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826314.354592:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826314.354607:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.354616:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.354623:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826314.354639:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.354647:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.354659:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826314.354663:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826314.354668:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.354674:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4aa23 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826314.354693:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346209280 00000100:00100000:2.0:1787826314.354704:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346209280 00000100:00100000:2.0:1787826314.354769:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11087 00000100:00100000:2.0:1787826314.354780:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3652:x1874663346209408:12345-0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826314.354801:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11088 00010000:00010000:2.0:1787826314.354827:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787826314.355089:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346209280:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826314.355127:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826314.355176:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aa15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x194:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4aa0e expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.355192:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x4ec94fd6 (ffff9f794385ad00) 00010000:00010000:0.0:1787826314.355199:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826314.355395:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346209280:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 582us (814us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826314.355409:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11088 00000100:00100000:3.0:1787826314.355719:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346209280:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826314.355893:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826314.356129:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78555e7a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826314.356153:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346209536:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787826314.356205:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826314.356308:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346209536 00000100:00100000:3.0:1787826314.356316:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346209536 00000100:00100000:3.0:1787826314.356378:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30826 00000100:00100000:3.0:1787826314.356389:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346209536:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1787826314.356416:0:8443:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1311 count 1 00010000:00010000:3.0:1787826314.356445:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.356467:0:8443:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.356476:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=1s v=(1 0) 00010000:00010000:3.0:1787826314.356480:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826314.356484:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.356517:0:8443:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.356533:0:8443:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 5 pid: 35096 timeout: 8226 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826314.356545:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346209920:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826314.356609:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1b40 going to sleep for 11 seconds 00000100:00080000:1.0:1787826314.356886:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826314.357534:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346209792 00000100:00100000:0.0:1787826314.357543:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346209792 00000100:00100000:0.0:1787826314.357644:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23655 00000100:00100000:0.0:1787826314.357654:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346209792:12345-0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787826314.357846:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4aa3f lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4aa38 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.357878:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:2.0:1787826314.357909:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826314.358412:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346209408:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 3630us (4992us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826314.358426:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11087 00000100:00100000:3.0:1787826314.358449:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843091180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346209408:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826314.358531:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346209792:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826314.358586:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346209792:12345-0@lo:1000:osp_up1-0.0 Request processed in 933us (2692us total) trans 4294985395 rc 0/0 00000100:00100000:0.0:1787826314.358595:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23655 00000100:00100000:3.0:1787826314.361511:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346209920:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826314.361544:0:8443:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826314.363743:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826314.364073:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332591488 00000100:00100000:1.0:1787826314.364083:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332591488 00010000:00010000:1.0:1787826314.364174:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 6 pid: 35096 timeout: 8226 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826314.364273:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21941 00000100:00100000:1.0:1787826314.364285:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332591488:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826314.364300:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826314.364307:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 6 pid: 35096 timeout: 8226 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.364322:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 6 pid: 35096 timeout: 8226 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.364350:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000632eea58/0xe21309aa81f4aa2a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x51f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e876080146b expref: 6 pid: 35096 timeout: 8226 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.364366:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826314.364370:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x51f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826314.364376:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.364387:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.364421:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826314.364484:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332591488:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 201us (749us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826314.364491:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21941 00010000:00010000:3.0:1787826314.364555:0:8443:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.364575:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.364586:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.364595:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.364618:0:8443:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826314.364825:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346209536:12345-0@lo:6:osp-syn-0-0.0 Request processed in 8437us (8619us total) trans 4294973848 rc 0/0 00000100:00100000:3.0:1787826314.364834:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30826 00010000:00010000:3.0:1787826314.364855:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826314.364861:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.364870:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826314.364874:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.364883:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.364911:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.364919:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4aa5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x51f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826314.364922:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f796cdb0a80/2, rc 0, transno 4294973848 00000100:00100000:0.0:1787826314.364933:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346209536:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1787826314.364950:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826314.364954:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826314.364957:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826314.412464:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826314.461504:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.461554:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.461589:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.461613:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.461650:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.461673:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.461691:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.461718:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.461748:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332591744 00000100:00100000:2.0:1787826314.461759:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332591744 00000100:00100000:2.0:1787826314.461810:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332592256 00000100:00100000:2.0:1787826314.461813:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332592256 00000100:00100000:2.0:1787826314.461829:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332591872 00000100:00100000:2.0:1787826314.461832:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332591872 00000100:00100000:2.0:1787826314.461849:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332592128 00000100:00100000:2.0:1787826314.461852:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332592128 00000100:00100000:2.0:1787826314.461875:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14290 00000100:00100000:2.0:1787826314.461885:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332591744:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826314.461976:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332591744:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 91us (492us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826314.461983:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14290 00000100:00100000:2.0:1787826314.462003:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14291 00000100:00100000:2.0:1787826314.462010:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332592256:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826314.462224:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14292 00000100:00100000:0.0:1787826314.462235:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332591872:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826314.462267:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332591872:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 36us (593us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826314.462274:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14292 00000100:00100000:0.0:1787826314.462290:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14293 00000100:00100000:0.0:1787826314.462297:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332592128:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826314.462335:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332592128:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 37us (642us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826314.462342:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14293 00000100:00100000:3.0:1787826314.462471:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332592000 00000100:00100000:3.0:1787826314.462476:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332592000 00000100:00100000:3.0:1787826314.462534:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332592512 00000100:00100000:3.0:1787826314.462538:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332592512 00000100:00100000:3.0:1787826314.462558:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332592384 00000100:00100000:3.0:1787826314.462561:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332592384 00000100:00100000:3.0:1787826314.462583:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8812 00000100:00100000:3.0:1787826314.462590:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332592000:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826314.462621:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332592000:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 31us (1066us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.462639:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8812 00000100:00100000:3.0:1787826314.462653:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8813 00000100:00100000:3.0:1787826314.462660:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332592512:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826314.462685:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332592512:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (1071us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.462691:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8813 00000100:00100000:3.0:1787826314.462702:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8814 00000100:00100000:3.0:1787826314.462708:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332592384:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826314.462735:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332592384:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (1015us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.462741:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8814 00000100:00080000:0.0:1787826314.462967:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826314.463536:0:30901:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332591616 00000100:00100000:3.0:1787826314.463543:0:30901:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332591616 00000100:00100000:3.0:1787826314.463764:0:30901:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2222 00000100:00100000:3.0:1787826314.463774:0:30901:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7974f69c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332591616:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826314.463814:0:30901:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7974f69c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332591616:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 42us (2163us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.463820:0:30901:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2222 00000100:00100000:2.0:1787826314.463929:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332592256:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 1910us (2330us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826314.463937:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14291 00000100:00100000:3.0:1787826314.861676:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.861900:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332592640 00000100:00100000:2.0:1787826314.861910:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332592640 00000100:00100000:2.0:1787826314.861975:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68417 00000100:00100000:2.0:1787826314.861986:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:224933:x1874663332592640:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826314.862008:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826314.862082:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aa62 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801479 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.862178:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4aa69 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.862191:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4aa69 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.862204:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4aa69 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826314.862301:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4c:0x0]: returning size 4096 00000004:00200000:2.0:1787826314.862306:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787826314.862325:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4aa69 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.862352:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aa62 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801479 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.862376:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4aa69 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801479 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.862384:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:2.0:1787826314.862387:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826314.862391:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826314.862461:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224933:x1874663332592640:12345-192.168.203.56@tcp:101:rm.0 Request processed in 477us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826314.862469:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68417 00000100:00100000:0.0:1787826314.869168:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826314.869214:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332592768 00000100:00100000:0.0:1787826314.869224:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332592768 00000100:00100000:0.0:1787826314.869276:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68418 00000100:00100000:0.0:1787826314.869286:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224933:x1874663332592768:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826314.869304:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826314.869367:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4aa70 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801480 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.869430:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.869449:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.869459:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826314.869508:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x194:0x0]: returning size 4096 00000004:00200000:0.0:1787826314.869513:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826314.869550:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4aa70 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.869567:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.869576:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826314.869579:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826314.869582:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826314.869652:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224933:x1874663332592768:12345-192.168.203.56@tcp:101:rm.0 Request processed in 367us (495us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826314.869659:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68418 00000100:00100000:3.0:1787826314.872338:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826314.873002:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332592896 00000100:00100000:1.0:1787826314.873013:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332592896 00000100:00100000:1.0:1787826314.873077:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68419 00000100:00100000:1.0:1787826314.873090:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224933:x1874663332592896:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826314.873140:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826314.873222:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4aa7e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801487 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.873286:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.873299:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.873312:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826314.873361:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x192:0x0]: returning size 4096 00000004:00200000:1.0:1787826314.873366:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826314.873404:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4aa7e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.873421:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826314.873428:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f794385a400) 00010000:00010000:1.0:1787826314.873431:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826314.873434:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826314.873506:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224933:x1874663332592896:12345-192.168.203.56@tcp:101:rm.0 Request processed in 417us (1175us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826314.873513:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68419 00000100:00100000:0.0:1787826314.878579:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826314.878632:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332593152 00000100:00100000:3.0:1787826314.878643:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332593152 00000100:00100000:3.0:1787826314.878709:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68420 00000100:00100000:3.0:1787826314.878721:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224933:x1874663332593152:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826314.878743:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826314.878765:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4aa8c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080148e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826314.878942:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4c:0x0]: returning size 4096 00010000:00010000:3.0:1787826314.879052:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4aa8c lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080148e expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.879066:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7843326d00) 00010000:00010000:3.0:1787826314.879070:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826314.879075:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4aa8c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080148e expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.879092:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826314.879305:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224933:x1874663332593152:12345-192.168.203.56@tcp:101:rm.0 Request processed in 584us (734us total) trans 4294985396 rc 301/301 00000100:00100000:3.0:1787826314.879313:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68420 00000100:00100000:3.0:1787826314.888598:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.890579:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332593280 00000100:00100000:2.0:1787826314.890595:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332593280 00000100:00100000:2.0:1787826314.890679:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9341 00000100:00100000:2.0:1787826314.890693:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224933:x1874663332593280:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826314.905212:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224933:x1874663332593280:12345-192.168.203.56@tcp:37:rm.0 Request processed in 14510us (16611us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826314.905228:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9341 00000100:00100000:3.0:1787826314.922370:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826314.922472:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332593408 00000100:00100000:0.0:1787826314.922482:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332593408 00000100:00100000:0.0:1787826314.922535:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9342 00000100:00100000:0.0:1787826314.922546:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224933:x1874663332593408:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826314.932766:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224933:x1874663332593408:12345-192.168.203.56@tcp:37:rm.0 Request processed in 10211us (10395us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.932781:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9342 00000100:00100000:1.0:1787826314.938210:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826314.939649:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332593664 00000100:00100000:3.0:1787826314.939981:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332593664 00000100:00100000:3.0:1787826314.940058:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9343 00000100:00100000:3.0:1787826314.940070:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:224933:x1874663332593664:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:3.0:1787826314.942148:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1199, 344 off1092072 00000040:00080000:3.0:1787826314.942207:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1517, 424 off1533576 00000004:00080000:3.0:1787826314.942213:0:17601:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d9b13c0 with version 6830 00000004:00080000:1.0:1787826314.942295:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6830 rpc_version 6830 00000020:00100000:1.0:1787826314.942332:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826314.942343:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc31500 x1874663346210176/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826314.942371:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346210176:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826314.942445:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826314.942485:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78555e76c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826314.942593:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346210176 00000100:00100000:3.0:1787826314.942601:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346210176 00000100:00100000:3.0:1787826314.942666:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23656 00000100:00100000:3.0:1787826314.942675:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346210176:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826314.942913:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346210176:12345-0@lo:1000:osp_up0-1.0 Request processed in 238us (467us total) trans 8589957389 rc 0/0 00000100:00100000:3.0:1787826314.942921:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23656 00000100:00100000:1.0:1787826314.943416:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957386 gen 2 00000004:00080000:1.0:1787826314.943422:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957385 committed transno 8589957386 00000004:00080000:1.0:1787826314.943440:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957386 committed transno 8589957386 00000100:00100000:1.0:1787826314.943453:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f784cc31500 x1874663346210176/t8589957389(8589957389) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826330 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826314.943479:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346210176:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826314.944043:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224933:x1874663332593664:12345-192.168.203.56@tcp:35:rm.0 Request processed in 3972us (5848us total) trans 4294985397 rc 0/0 00000100:00100000:3.0:1787826314.944052:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9343 00000100:00100000:1.0:1787826314.953838:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826314.954034:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332593920 00000100:00100000:3.0:1787826314.954045:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332593920 00000100:00100000:3.0:1787826314.954131:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68421 00000100:00100000:3.0:1787826314.954143:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967913100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224933:x1874663332593920:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826314.954169:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826314.954192:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4aa9a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801495 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.954361:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4aaa1 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.954375:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4aaa1 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.954387:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4aaa1 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826314.954398:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4c:0x0]: returning size 4096 00010000:00010000:3.0:1787826314.954502:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4aa9a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801495 expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.954527:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4aaa1 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801495 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.954536:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:3.0:1787826314.954538:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1787826314.954542:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826314.954601:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967913100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224933:x1874663332593920:12345-192.168.203.56@tcp:101:rm.0 Request processed in 461us (773us total) trans 4294985398 rc 0/0 00000100:00100000:3.0:1787826314.954609:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68421 00000100:00100000:1.0:1787826314.965170:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826314.965243:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332594048 00000100:00100000:3.0:1787826314.965386:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332594048 00000100:00100000:3.0:1787826314.965444:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68422 00000100:00100000:3.0:1787826314.965454:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224933:x1874663332594048:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:3.0:1787826314.965504:0:35080:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:3.0:1787826314.966140:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224933:x1874663332594048:12345-192.168.203.56@tcp:41:rm.0 Request processed in 676us (968us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.966837:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68422 00000100:00100000:3.0:1787826314.972708:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.973145:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332594176 00000100:00100000:2.0:1787826314.973157:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332594176 00000100:00100000:2.0:1787826314.973221:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68423 00000100:00100000:2.0:1787826314.973231:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:224933:x1874663332594176:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826314.973252:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826314.973275:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4aaaf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080149c expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973351:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973366:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973377:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973392:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973399:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973406:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973487:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4aac4 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973495:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4aac4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973501:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4aac4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826314.973522:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4aac4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973538:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973545:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973551:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973558:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973564:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826314.973569:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973576:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973589:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f794385a400) 00010000:00010000:2.0:1787826314.973592:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826314.973597:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973602:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4aab6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973614:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973620:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973636:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973640:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826314.973643:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973649:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973656:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f79785cdc00) 00010000:00010000:2.0:1787826314.973658:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826314.973661:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973667:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aabd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973702:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4aaaf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080149c expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973715:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4aac4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080149c expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.973722:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f79785cdd00) 00010000:00010000:2.0:1787826314.973725:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826314.973727:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826314.973795:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224933:x1874663332594176:12345-192.168.203.56@tcp:101:rm.0 Request processed in 565us (1096us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826314.974056:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68423 00000100:00100000:2.0:1787826314.980972:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826314.981529:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332594304 00000100:00100000:3.0:1787826314.981540:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332594304 00000100:00100000:3.0:1787826314.981604:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68424 00000100:00100000:3.0:1787826314.981635:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:224933:x1874663332594304:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826314.981661:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826314.981688:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4aacb lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014a3 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.981756:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4aad2 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.981770:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4aad2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.981781:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4aad2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826314.981836:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4d:0x0]: returning size 6 00000004:00200000:3.0:1787826314.981841:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:3.0:1787826314.981880:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4aacb lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014a3 expref: 14 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.981899:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4aad2 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014a3 expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.981908:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f7843326d00) 00010000:00010000:3.0:1787826314.981911:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826314.981915:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826314.981996:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224933:x1874663332594304:12345-192.168.203.56@tcp:101:rm.0 Request processed in 371us (1033us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.982003:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68424 00000100:00100000:2.0:1787826314.990012:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826314.990061:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826314.990734:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332594432 00000100:00100000:3.0:1787826314.990746:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332594432 00000100:00100000:3.0:1787826314.990823:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11089 00000100:00100000:3.0:1787826314.990834:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332594432:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826314.990851:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826314.990878:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4aac4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080149c expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.990896:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f79785cdd00) 00010000:00010000:3.0:1787826314.990909:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826314.990976:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332594432:12345-192.168.203.56@tcp:103:rm.0 Request processed in 143us (971us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.990987:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11089 00000100:00100000:0.0:1787826314.991072:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332594688 00000100:00100000:0.0:1787826314.991078:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332594688 00000100:00100000:0.0:1787826314.991130:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68425 00000100:00100000:0.0:1787826314.991138:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224933:x1874663332594688:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826314.991191:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826314.991208:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4aad2 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014a3 expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.991221:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f7843326d00) 00010000:00010000:0.0:1787826314.991229:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826314.991279:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.991290:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.991300:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826314.991324:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093b80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346210304:0@lo:101:mdt00_011.0 00000100:00100000:0.0:1787826314.991371:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826314.991404:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826314.991654:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346210304 00000100:00100000:2.0:1787826314.991660:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346210304 00000100:00100000:2.0:1787826314.991699:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23657 00000100:00100000:2.0:1787826314.991707:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35094:x1874663346210304:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826314.991720:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826314.991734:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4aad9 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.991756:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.991764:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aad9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.991794:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826314.991820:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 9 pid: 35093 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826314.991831:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98ca80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346210432:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826314.991890:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548af00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826314.996382:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826314.996451:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332594816 00000100:00100000:0.0:1787826314.996460:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332594816 00010000:00010000:0.0:1787826314.996501:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 10 pid: 35093 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826314.996515:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98ca80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346210432:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826314.996542:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21942 00010000:00010000:3.0:1787826314.996544:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aad9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826314.996551:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332594816:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826314.996555:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f794385a400) 00010000:00010000:3.0:1787826314.996558:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826314.996562:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aad9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.996564:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826314.996571:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 10 pid: 35093 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.996572:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00010000:00010000:0.0:1787826314.996582:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 10 pid: 35093 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.996604:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4aa85 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x192:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801487 expref: 10 pid: 35093 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.996618:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f794385a400) 00010000:00010000:0.0:1787826314.996631:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aad9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.996642:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4aad9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.996651:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4aad9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826314.996651:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35094:x1874663346210304:12345-0@lo:101:mdt00_011.0 Request processed in 4945us (5278us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.996658:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23657 00010000:00010000:0.0:1787826314.996683:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aad9 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826314.996699:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346210560:0@lo:105: 00000100:00100000:0.0:1787826314.996738:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826314.996774:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346210560 00000100:00100000:0.0:1787826314.996777:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826314.996778:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346210560 00000100:00100000:0.0:1787826314.996802:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093b80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346210304:0@lo:101:mdt00_011.0 00000100:00100000:3.0:1787826314.996808:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7861 00000100:00100000:3.0:1787826314.996815:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346210560:12345-0@lo:105: 00010000:00010000:3.0:1787826314.996826:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.996835:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.996844:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.996856:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.996864:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.996872:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.996884:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42000) 00010000:00010000:0.0:1787826314.996889:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826314.996915:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x192:0x0] 00000100:00100000:3.0:1787826314.996916:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346210560:12345-0@lo:105: Request processed in 101us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.996921:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7861 00010000:00010000:0.0:1787826314.997074:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.997082:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.997092:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826314.997299:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346210688:0@lo:101:mdt00_011.0 00000100:00100000:0.0:1787826314.997329:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826314.997355:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826314.997391:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346210560:0@lo:105: 00010000:00010000:0.0:1787826314.997401:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826314.997495:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346210688 00000100:00100000:3.0:1787826314.997500:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346210688 00000100:00100000:3.0:1787826314.997527:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23658 00000100:00100000:3.0:1787826314.997534:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:35094:x1874663346210688:12345-0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826314.997548:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826314.997563:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4aaee lrc: 2/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4aae7 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.997580:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4aaee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aae7 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.997593:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4aaee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aae7 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826314.997615:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f7843326600) 00010000:00010000:3.0:1787826314.997618:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826314.997620:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826314.997654:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346210688:12345-0@lo:101:mdt00_011.0 Request processed in 121us (324us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826314.997659:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23658 00000100:00100000:0.0:1787826314.999553:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332594816:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2994us (3174us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826314.999564:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21942 00000100:00100000:0.0:1787826314.999650:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346210688:0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826314.999675:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aaee expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826314.999689:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aaee expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826314.999713:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x192:0x0] 00000100:00100000:0.0:1787826314.999841:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346210816:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826315.000567:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.000605:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 10 seconds 00000100:00100000:3.0:1787826315.003768:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346210816 00000100:00100000:3.0:1787826315.003785:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346210816 00000100:00100000:3.0:1787826315.003866:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23659 00000100:00100000:3.0:1787826315.003877:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346210816:12345-0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826315.004007:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346210816:12345-0@lo:1000:mdt00_011.0 Request processed in 132us (3441us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.004017:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23659 00000100:00100000:3.0:1787826315.004089:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346210816:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826315.004198:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346210944:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826315.004244:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.004279:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e810c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.004320:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346210944 00000100:00100000:2.0:1787826315.004326:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346210944 00000100:00100000:2.0:1787826315.004370:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23660 00000100:00100000:2.0:1787826315.004378:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346210944:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826315.004450:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346210944:12345-0@lo:1000:mdt00_011.0 Request processed in 73us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.004456:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23660 00000100:00100000:3.0:1787826315.004533:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346210944:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826315.004582:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465cbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346211072:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826315.004607:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.004636:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e810c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.004682:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346211072 00000100:00100000:2.0:1787826315.004686:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346211072 00000100:00100000:2.0:1787826315.004717:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23661 00000100:00100000:2.0:1787826315.004724:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346211072:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826315.004799:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346211072:12345-0@lo:1000:mdt00_011.0 Request processed in 76us (190us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826315.004818:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23661 00000100:00100000:3.0:1787826315.004883:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465cbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346211072:0@lo:1000:mdt00_011.0 00010000:00010000:3.0:1787826315.004956:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.004968:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.004979:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.004992:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465cbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346211200:0@lo:101:mdt00_011.0 00000100:00100000:3.0:1787826315.005019:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.005041:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.005092:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346211200 00000100:00100000:2.0:1787826315.005097:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346211200 00000100:00100000:2.0:1787826315.005147:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23662 00000100:00100000:2.0:1787826315.005154:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346211200:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826315.005164:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826315.005181:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aafc lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4aaf5 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.005216:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aafc lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aaf5 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.005231:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aafc lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aaf5 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.005240:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f79785cdc00) 00010000:00010000:2.0:1787826315.005244:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826315.005247:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826315.005288:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346211200:12345-0@lo:101:mdt00_011.0 Request processed in 134us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.005294:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23662 00000100:00100000:3.0:1787826315.005532:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465cbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346211200:0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826315.005545:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.005556:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.005679:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.005689:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.005698:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787826315.006330:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].222, 64 off22400 00000004:00080000:3.0:1787826315.006344:0:35094:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8127218237561ns, next at 8128218234859ns, rc = 0 00000040:00080000:3.0:1787826315.006374:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1200, 384 off1092456 00000040:00080000:3.0:1787826315.006414:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1518, 464 off1534040 00000004:00080000:3.0:1787826315.006419:0:35094:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81900 with version 6831 00000004:00080000:1.0:1787826315.007405:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6831 rpc_version 6831 00000020:00100000:1.0:1787826315.007538:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826315.007550:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7967910a80 x1874663346211456/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1664/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826315.007573:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967910a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346211456:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826315.007725:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.007761:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.007882:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346211456 00000100:00100000:2.0:1787826315.007890:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346211456 00000100:00100000:2.0:1787826315.007988:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23663 00000100:00100000:2.0:1787826315.007998:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346211456:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826315.012351:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346211456:12345-0@lo:1000:osp_up0-1.0 Request processed in 4345us (4616us total) trans 8589957390 rc 0/0 00000100:00100000:2.0:1787826315.012368:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23663 00000100:00100000:1.0:1787826315.012518:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967910a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346211456:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826315.012714:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012737:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012750:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012759:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012766:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012772:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012796:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787826315.012828:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012835:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012842:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012847:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826315.012853:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012860:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012877:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f7843326900) 00010000:00010000:2.0:1787826315.012880:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826315.012884:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012890:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ab03 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012903:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012912:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012918:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012937:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826315.012955:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x192:0x0] 00010000:00010000:2.0:1787826315.012964:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.012965:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826315.012977:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.012983:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.012986:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.012997:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826315.013002:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1787826315.013003:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826315.013010:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x192:0x0] 00010000:00010000:0.0:1787826315.013015:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.013018:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aaee expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.013028:0:35094:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aaee expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.013043:0:30445:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f7843326900) 00010000:00010000:0.0:1787826315.013047:0:30445:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826315.013050:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.013057:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ab0a lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e4d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.013076:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826315.013080:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.013086:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826315.013091:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826315.013099:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826315.013139:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224933:x1874663332594688:12345-192.168.203.56@tcp:36:rm.0 Request processed in 21967us (23043us total) trans 4294985399 rc 0/0 00000100:00100000:2.0:1787826315.013148:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68425 00010000:00010000:0.0:1787826315.013156:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.013165:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826315.013177:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826315.013184:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.013190:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.013193:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826315.013200:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.013204:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4aaf5 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aafc expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826315.013219:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.013226:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826315.013230:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826315.013232:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.013242:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.013250:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826315.013258:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.013265:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4aad9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aae0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.013281:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.013282:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.013284:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826315.013407:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826315.013436:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346211584:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826315.013488:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.013526:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826315.013533:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346211712:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826315.013557:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.013589:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346211584 00000100:00100000:1.0:1787826315.013595:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346211584 00000100:00100000:1.0:1787826315.013645:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346211712 00000100:00100000:1.0:1787826315.013649:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346211712 00000100:00100000:1.0:1787826315.013672:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11090 00000100:00100000:1.0:1787826315.013680:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971596c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346211584:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826315.013691:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826315.013709:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4aafc lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4aaf5 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.013721:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4d:0x0].0x0 (ffff9f79785cdc00) 00010000:00010000:1.0:1787826315.013729:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826315.013758:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971596c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346211584:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.013764:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11090 00000100:00100000:1.0:1787826315.013899:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11091 00000100:00100000:1.0:1787826315.013908:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346211712:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826315.013916:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826315.013928:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aae0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x192:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4aad9 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.013938:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f794385a400) 00010000:00010000:1.0:1787826315.013945:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826315.013971:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346211712:12345-0@lo:103:ldlm_bl.0 Request processed in 65us (413us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.013977:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11091 00000100:00100000:1.0:1787826315.014013:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346211712:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826315.014022:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346211584:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826315.018361:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.018406:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.018442:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332595200 00000100:00100000:0.0:1787826315.018443:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332594944 00000100:00100000:0.0:1787826315.018453:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332594944 00000100:00100000:3.0:1787826315.018453:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332595200 00000100:00100000:0.0:1787826315.018530:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11092 00000100:00100000:0.0:1787826315.018542:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332594944:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826315.018557:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826315.018584:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4a935 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080141e expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.018603:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:0.0:1787826315.018608:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826315.018680:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332594944:12345-192.168.203.56@tcp:103:rm.0 Request processed in 141us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.018687:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11092 00000100:00080000:0.0:1787826315.024387:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787826315.024398:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826315.033015:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9344 00000100:00100000:2.0:1787826315.033035:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:224933:x1874663332595200:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:2.0:1787826315.033415:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:224933:x1874663332595200:12345-192.168.203.56@tcp:35:rm.0 Request processed in 387us (15007us total) trans 4294985400 rc 0/0 00000100:00100000:2.0:1787826315.033424:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9344 00000100:00100000:3.0:1787826315.043471:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.043511:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332595456 00000100:00100000:3.0:1787826315.043522:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332595456 00000100:00100000:3.0:1787826315.043580:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68426 00000100:00100000:3.0:1787826315.043590:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:224933:x1874663332595456:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826315.043665:0:35093:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826315.043694:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4aaa1 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801495 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.043715:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4aa69 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801479 expref: 13 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.043730:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4a943 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801425 expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.043742:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:3.0:1787826315.043748:0:35093:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826315.043796:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.047754:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.047781:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.047827:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346211840:0@lo:101:mdt00_010.0 00000100:00100000:3.0:1787826315.047904:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.047943:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e813c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826315.047979:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346211840 00000100:00100000:3.0:1787826315.047987:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346211840 00000100:00100000:3.0:1787826315.048041:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23664 00000100:00100000:3.0:1787826315.048050:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346211840:12345-0@lo:101:mdt00_010.0 00010000:00010000:3.0:1787826315.048070:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826315.048090:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ab11 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.048133:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.048143:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.048152:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.048181:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.048194:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 7 pid: 11629 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.048205:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346211968:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826315.048283:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.048292:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 7 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.048301:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346212096:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826315.048326:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826315.054686:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.054711:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.054761:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346211968:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826315.054774:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346212096:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826315.054881:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.054967:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826315.054970:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826315.054975:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.054987:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826315.055044:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35093:x1874663346211840:12345-0@lo:101:mdt00_010.0 Request processed in 6994us (7140us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.055053:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23664 00000100:00100000:3.0:1787826315.055122:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346211840:0@lo:101:mdt00_010.0 00010000:00010000:3.0:1787826315.055141:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055152:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.055175:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332595712 00000100:00100000:3.0:1787826315.055180:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332595712 00010000:00010000:3.0:1787826315.055209:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 8 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.055234:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332595584 00000100:00100000:3.0:1787826315.055237:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332595584 00010000:00010000:3.0:1787826315.055262:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 8 pid: 11629 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.055282:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21943 00000100:00100000:3.0:1787826315.055290:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332595712:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826315.055303:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826315.055308:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 8 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055318:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 8 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055337:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4a919 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801417 expref: 8 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055347:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826315.055352:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055359:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826315.055393:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332595712:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 104us (716us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.055399:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21943 00000100:00100000:3.0:1787826315.055411:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21944 00000100:00100000:3.0:1787826315.055417:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332595584:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826315.055424:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826315.055429:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 8 pid: 11629 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055439:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 8 pid: 11629 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055451:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4a982 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801441 expref: 8 pid: 11629 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055460:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826315.055465:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055473:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055481:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.055509:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.055523:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346212224:0@lo:105: 00000100:00100000:3.0:1787826315.055556:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.055591:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e813c0 going to sleep for 11 seconds 00000100:00080000:3.0:1787826315.055604:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826315.065486:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346212224 00000100:00100000:1.0:1787826315.065501:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346212224 00000100:00100000:1.0:1787826315.065851:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7862 00000100:00100000:1.0:1787826315.065924:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346212224:12345-0@lo:105: 00010000:00010000:1.0:1787826315.065944:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.065957:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.065967:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.065977:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.065986:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.065995:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.066030:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68800) 00000100:00100000:1.0:1787826315.066157:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346212224:12345-0@lo:105: Request processed in 239us (10598us total) trans 0 rc 0/0 00010000:00010000:3.0:1787826315.066168:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.066169:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7862 00010000:00010000:3.0:1787826315.066184:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826315.066215:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787826315.066227:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346212224:0@lo:105: 00010000:00010000:1.0:1787826315.066247:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826315.066261:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.066270:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.066282:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.066297:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332595584:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 10877us (11583us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.066304:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346212352:0@lo:101:mdt00_010.0 00000100:00100000:1.0:1787826315.066311:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21944 00000100:00100000:3.0:1787826315.066350:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.066377:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.070988:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346212352 00000100:00100000:2.0:1787826315.071002:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346212352 00000100:00100000:2.0:1787826315.071066:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23665 00000100:00100000:2.0:1787826315.071077:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:35093:x1874663346212352:12345-0@lo:101:mdt00_010.0 00010000:00010000:2.0:1787826315.071098:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826315.071198:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ab26 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ab1f expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.071229:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ab26 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ab1f expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.071243:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ab26 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab1f expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.071252:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cdc00) 00010000:00010000:2.0:1787826315.071255:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826315.071259:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826315.071344:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35093:x1874663346212352:12345-0@lo:101:mdt00_010.0 Request processed in 269us (4993us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.071351:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23665 00000100:00100000:0.0:1787826315.071725:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346212352:0@lo:101:mdt00_010.0 00010000:00010000:0.0:1787826315.071745:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.071757:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826315.071780:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826315.071829:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346212480:0@lo:1000:mdt00_010.0 00000100:00100000:0.0:1787826315.071871:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.071903:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826315.072134:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346212480 00000100:00100000:1.0:1787826315.072141:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346212480 00000100:00100000:1.0:1787826315.072177:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23666 00000100:00100000:1.0:1787826315.072185:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35093:x1874663346212480:12345-0@lo:1000:mdt00_010.0 00000100:00100000:1.0:1787826315.072266:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35093:x1874663346212480:12345-0@lo:1000:mdt00_010.0 Request processed in 81us (394us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.072274:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23666 00000100:00100000:0.0:1787826315.072725:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346212480:0@lo:1000:mdt00_010.0 00000100:00100000:0.0:1787826315.072775:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346212608:0@lo:1000:mdt00_010.0 00000100:00100000:0.0:1787826315.072801:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.072826:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.072851:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346212608 00000100:00100000:2.0:1787826315.072857:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346212608 00000100:00100000:2.0:1787826315.072960:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23667 00000100:00100000:2.0:1787826315.072970:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35093:x1874663346212608:12345-0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826315.073085:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35093:x1874663346212608:12345-0@lo:1000:mdt00_010.0 Request processed in 116us (281us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826315.073093:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23667 00000100:00100000:1.0:1787826315.073261:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346212608:0@lo:1000:mdt00_010.0 00010000:00010000:1.0:1787826315.073329:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.073340:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.073349:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.073364:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346212736:0@lo:101:mdt00_010.0 00000100:00100000:1.0:1787826315.073400:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.073498:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826315.074218:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346212736 00000100:00100000:0.0:1787826315.074225:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346212736 00000100:00100000:0.0:1787826315.074272:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23668 00000100:00100000:0.0:1787826315.074280:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35093:x1874663346212736:12345-0@lo:101:mdt00_010.0 00010000:00010000:0.0:1787826315.074293:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826315.074310:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ab2d expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.074588:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.074606:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ab2d expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.074647:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.074662:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 6 pid: 35080 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826315.074675:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346212864:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826315.075288:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d76c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826315.078278:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346212864:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826315.078315:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab2d expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.078328:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:1.0:1787826315.078331:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826315.078335:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab2d expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.078345:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:1.0:1787826315.078426:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:35093:x1874663346212736:12345-0@lo:101:mdt00_010.0 Request processed in 4145us (5024us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.078434:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23668 00000100:00100000:3.0:1787826315.078578:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346212736:0@lo:101:mdt00_010.0 00010000:00010000:3.0:1787826315.078599:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.078612:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.079252:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826315.080491:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332595840 00000100:00100000:0.0:1787826315.080502:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332595840 00010000:00010000:0.0:1787826315.080544:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 7 pid: 35080 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826315.080614:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21945 00000100:00100000:0.0:1787826315.080637:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332595840:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826315.080652:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826315.080659:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 7 pid: 35080 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.080673:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 7 pid: 35080 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.080701:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4a99e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801448 expref: 7 pid: 35080 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.080714:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:0.0:1787826315.080720:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab2d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.080730:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4ab2d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.080738:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4ab2d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.080769:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab2d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826315.080783:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346212992:0@lo:105: 00000100:00100000:0.0:1787826315.080840:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.080887:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826315.080998:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346212992 00000100:00100000:3.0:1787826315.081004:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346212992 00000100:00100000:3.0:1787826315.081041:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7863 00000100:00100000:3.0:1787826315.081050:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346212992:12345-0@lo:105: 00010000:00010000:3.0:1787826315.081061:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.081072:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.081080:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.081087:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.081094:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.081102:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.081340:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf200) 00010000:00010000:0.0:1787826315.081453:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081463:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081503:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081514:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081522:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081541:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081551:0:35093:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081558:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081582:0:35093:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.081593:0:35093:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 11 pid: 35094 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826315.081605:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001_UUID:35093:1874663346213120:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826315.081677:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d79c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826315.082327:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346212992:12345-0@lo:105: Request processed in 1275us (1482us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.082336:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7863 00000100:00100000:0.0:1787826315.082830:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346212992:0@lo:105: 00010000:00010000:0.0:1787826315.082850:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826315.082891:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332595840:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2267us (3642us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.084277:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21945 00000100:00100000:0.0:1787826315.084686:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001_UUID:35093:1874663346213120:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826315.084709:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826315.086017:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.086227:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332595968 00000100:00100000:3.0:1787826315.086237:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332595968 00010000:00010000:3.0:1787826315.086277:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 12 pid: 35094 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.086353:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21946 00000100:00100000:3.0:1787826315.086364:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332595968:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826315.086379:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826315.086386:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 12 pid: 35094 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.086399:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 12 pid: 35094 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.086422:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4aa77 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801480 expref: 12 pid: 35094 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.086438:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:3.0:1787826315.086443:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.086452:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.086460:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.086490:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826315.086546:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332595968:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 183us (539us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.086554:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21946 00010000:00010000:0.0:1787826315.086601:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.086609:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.086663:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.086671:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.086680:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826315.086701:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346213248:0@lo:101:mdt00_010.0 00000100:00100000:0.0:1787826315.086747:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.086786:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d79c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.086827:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346213248 00000100:00100000:2.0:1787826315.086832:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346213248 00000100:00100000:2.0:1787826315.086871:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23669 00000100:00100000:2.0:1787826315.086879:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:35093:x1874663346213248:12345-0@lo:101:mdt00_010.0 00010000:00010000:2.0:1787826315.087130:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826315.087287:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab50 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x192:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ab49 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.087310:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab50 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ab49 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.087322:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab50 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab49 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.087330:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f79785cdd00) 00010000:00010000:2.0:1787826315.087333:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826315.087335:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826315.087562:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35093:x1874663346213248:12345-0@lo:101:mdt00_010.0 Request processed in 683us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.087569:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23669 00000100:00100000:0.0:1787826315.087648:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346213248:0@lo:101:mdt00_010.0 00010000:00010000:0.0:1787826315.087664:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.087674:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826315.088717:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346213376:0@lo:403:mdt00_010.0 00000100:00100000:2.0:1787826315.088777:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.088853:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826315.088895:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346213376 00000100:00100000:3.0:1787826315.088902:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346213376 00000100:00100000:3.0:1787826315.089045:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23670 00000100:00100000:3.0:1787826315.089058:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:35093:x1874663346213376:12345-0@lo:403:mdt00_010.0 00000100:00100000:2.0:1787826315.094801:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:35093:x1874663346213376:12345-0@lo:403:mdt00_010.0 Request processed in 5737us (6014us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.094817:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23670 00000100:00100000:3.0:1787826315.096153:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346213376:0@lo:403:mdt00_010.0 00000040:00080000:3.0:1787826315.097215:0:35093:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1201, 952 off1093408 00000040:00080000:3.0:1787826315.097395:0:35093:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1519, 1032 off1535072 00000004:00080000:3.0:1787826315.097401:0:35093:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81180 with version 6832 00000004:00080000:1.0:1787826315.097593:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6832 rpc_version 6832 00000020:00100000:1.0:1787826315.097642:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826315.097652:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9eaa00 x1874663346213632/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826315.097676:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346213632:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826315.097746:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.097775:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846fba840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826315.098073:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346213632 00000100:00100000:0.0:1787826315.098082:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346213632 00000100:00100000:0.0:1787826315.098226:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23671 00000100:00100000:0.0:1787826315.098236:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346213632:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826315.098684:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346213632:12345-0@lo:1000:osp_up0-1.0 Request processed in 448us (936us total) trans 8589957391 rc 0/0 00000100:00100000:0.0:1787826315.098692:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23671 00000100:00100000:1.0:1787826315.099084:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346213632:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787826315.099167:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4c:0x0]: returning size 4096 00010000:00010000:3.0:1787826315.099191:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099201:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099210:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099216:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826315.099221:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099228:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099247:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x194:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:3.0:1787826315.099250:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826315.099254:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099259:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4ab42 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099281:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099289:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099296:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099315:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826315.099324:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x192:0x0] 00010000:00010000:3.0:1787826315.099332:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099339:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099349:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099353:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826315.099356:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099362:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099379:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:3.0:1787826315.099381:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826315.099384:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099390:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4ab3b lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099400:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099406:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099413:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099426:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787826315.099436:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826315.099440:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099447:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099453:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099476:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787826315.099480:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826315.099485:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.099493:0:35093:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.099568:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:224933:x1874663332595456:12345-192.168.203.56@tcp:36:rm.0 Request processed in 55978us (56105us total) trans 4294985401 rc 0/0 00000100:00100000:3.0:1787826315.099576:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68426 00010000:00010000:1.0:1787826315.099858:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787826315.099864:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.099872:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826315.099877:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826315.099885:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.099915:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.099922:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826315.100042:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346213760:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826315.100098:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.100187:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346213760 00000100:00100000:3.0:1787826315.100195:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346213760 00000100:00100000:3.0:1787826315.100239:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11093 00000100:00100000:3.0:1787826315.100248:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971597c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346213760:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826315.100258:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826315.100277:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab50 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ab49 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.100291:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x0 (ffff9f79785cdd00) 00010000:00010000:3.0:1787826315.100301:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826315.100333:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971597c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+14:3652:x1874663346213760:12345-0@lo:103:ldlm_bl.0 Request processed in 87us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.100338:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11093 00000100:00100000:3.0:1787826315.100401:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346213760:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826315.100514:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100526:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4ab49 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x192:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab50 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100542:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826315.100547:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100553:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826315.100558:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826315.100565:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100585:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100592:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826315.100610:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100618:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf200/0xe21309aa81f4ab2d lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab34 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100640:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826315.100644:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100650:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826315.100653:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826315.100659:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100677:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100683:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826315.100696:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100703:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4ab11 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab18 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100712:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826315.100714:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826315.100716:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826315.100723:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826315.100732:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346214016:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826315.100762:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.100792:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346214016 00000100:00100000:1.0:1787826315.100797:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346214016 00000100:00100000:1.0:1787826315.100825:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11094 00000100:00100000:1.0:1787826315.100832:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+14:3655:x1874663346214016:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826315.100839:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826315.100851:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ab18 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ab11 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.100862:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826315.100865:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826315.100893:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346214016:12345-0@lo:103:ldlm_bl.0 Request processed in 62us (130us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.100900:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11094 00000100:00100000:1.0:1787826315.100926:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346214016:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826315.101056:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826315.101305:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826315.101315:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346213888:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826315.101344:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.102068:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346213888 00000100:00100000:2.0:1787826315.102075:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346213888 00000100:00100000:2.0:1787826315.102154:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11095 00000100:00100000:2.0:1787826315.102164:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346213888:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826315.102176:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826315.102195:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ab34 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ab2d expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.102212:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f7845cb1300) 00010000:00010000:2.0:1787826315.102221:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826315.102274:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346213888:12345-0@lo:103:ldlm_bl.0 Request processed in 111us (929us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.102281:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11095 00010000:00010000:0.0:1787826315.102582:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826315.102674:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346213888:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826315.171800:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826315.171986:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332596096 00000100:00100000:2.0:1787826315.171997:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332596096 00000100:00100000:2.0:1787826315.172064:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68427 00000100:00100000:2.0:1787826315.172076:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224941:x1874663332596096:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826315.172145:0:17813:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:2.0:1787826315.172234:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224941:x1874663332596096:12345-192.168.203.56@tcp:41:stat.0 Request processed in 160us (444us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.172242:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68427 00000100:00100000:2.0:1787826315.214314:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.214578:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332596224 00000100:00100000:3.0:1787826315.214589:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332596224 00000100:00100000:3.0:1787826315.214670:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68428 00000100:00100000:3.0:1787826315.214682:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224945:x1874663332596224:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826315.214708:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826315.214733:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ab57 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014aa expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.214812:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.214827:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.214840:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826315.215027:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826315.215032:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826315.215057:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.215085:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ab57 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.215311:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.215322:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826315.215325:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826315.215328:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826315.215429:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224945:x1874663332596224:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 749us (1125us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.215436:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68428 00000100:00100000:0.0:1787826315.225699:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.225938:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332596352 00000100:00100000:3.0:1787826315.225949:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332596352 00000100:00100000:3.0:1787826315.226015:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68429 00000100:00100000:3.0:1787826315.226026:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224945:x1874663332596352:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826315.226047:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826315.226070:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ab65 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014b1 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.226186:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.226201:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.226212:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.226224:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.226234:0:35080:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ab26 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab1f expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.226248:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.226282:0:35080:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ab26 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4ab1f expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.226300:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000_UUID:35080:1874663346214144:0@lo:104: 00000100:00100000:3.0:1787826315.226352:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.226389:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826315.226635:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346214144 00000100:00100000:1.0:1787826315.226640:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346214144 00000100:00100000:1.0:1787826315.226678:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7864 00000100:00100000:1.0:1787826315.226685:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35080:x1874663346214144:12345-0@lo:104: 00010000:00010000:1.0:1787826315.226698:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.226765:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826315.226772:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35080:x1874663346214144:12345-0@lo:104: Request processed in 87us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.226778:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7864 00010000:00010000:2.0:1787826315.226841:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826315.226848:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.226856:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826315.226862:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826315.226870:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.226908:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.226915:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826315.226941:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.226948:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ab1f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ab26 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.226967:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.226968:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.226970:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826315.227156:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346214272:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826315.227192:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.227372:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346214272 00000100:00100000:2.0:1787826315.227376:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346214272 00010000:00010000:2.0:1787826315.227398:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ab26 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4ab1f expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826315.227430:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21947 00000100:00100000:2.0:1787826315.227438:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346214272:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826315.227448:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826315.229013:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000_UUID:35080:1874663346214144:0@lo:104: 00010000:00010000:3.0:1787826315.229040:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826315.230138:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957391 is committed 00010000:00010000:2.0:1787826315.230386:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ab26 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4ab1f expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.230404:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cdc00) 00010000:00010000:2.0:1787826315.230409:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.230418:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.230426:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.230457:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826315.230541:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346214272:12345-0@lo:103:ldlm_bl.0 Request processed in 3103us (3347us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.230552:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21947 00000100:00100000:1.0:1787826315.230656:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346214272:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826315.230731:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230741:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230804:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230813:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230820:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230827:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230834:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826315.230839:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230846:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230855:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826315.230858:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826315.230862:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230869:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ab6c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230879:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230885:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230903:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230908:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826315.230911:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230917:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230929:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cdc00) 00010000:00010000:3.0:1787826315.230931:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826315.230934:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230940:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4ab73 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230984:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ab65 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014b1 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.230995:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826315.230997:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826315.231001:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ab65 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014b1 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.231010:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826315.231071:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224945:x1874663332596352:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 5047us (5382us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826315.231080:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68429 00000100:00100000:0.0:1787826315.236061:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826315.236272:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332596480 00000100:00100000:2.0:1787826315.236282:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332596480 00000100:00100000:2.0:1787826315.236352:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68430 00000100:00100000:2.0:1787826315.236363:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:224945:x1874663332596480:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826315.236391:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826315.236415:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ab7a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014b8 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.236485:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.236499:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.236510:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826315.236560:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826315.236564:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826315.236584:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.236611:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ab7a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.236643:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.236653:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826315.236656:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826315.236660:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826315.236717:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:224945:x1874663332596480:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 355us (665us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.236725:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68430 00000100:00100000:2.0:1787826315.240827:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826315.240921:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332596736 00000100:00100000:0.0:1787826315.240930:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332596736 00000100:00100000:0.0:1787826315.241010:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68431 00000100:00100000:0.0:1787826315.241020:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224945:x1874663332596736:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:0.0:1787826315.241173:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346214400:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826315.241228:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.241260:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7540 going to sleep for 11 seconds 00000100:00080000:3.0:1787826315.241489:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826315.241549:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346214400 00000100:00100000:2.0:1787826315.241554:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346214400 00000100:00100000:2.0:1787826315.241595:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23672 00000100:00100000:2.0:1787826315.241603:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346214400:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826315.241700:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346214400:12345-0@lo:1000:mdt00_012.0 Request processed in 97us (470us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.241708:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23672 00000100:00100000:3.0:1787826315.241800:0:35095:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957391 gen 2 00000004:00080000:3.0:1787826315.241806:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957389 committed transno 8589957391 00000004:00080000:3.0:1787826315.241827:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957390 committed transno 8589957391 00000004:00080000:3.0:1787826315.241839:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957391 committed transno 8589957391 00000100:00100000:3.0:1787826315.241855:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346214400:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826315.241903:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346214528:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826315.241934:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.241953:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826315.242010:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346214528 00000100:00100000:1.0:1787826315.242015:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346214528 00000100:00100000:1.0:1787826315.242053:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23673 00000100:00100000:1.0:1787826315.242062:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346214528:12345-0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826315.242197:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346214528:12345-0@lo:1000:mdt00_012.0 Request processed in 137us (262us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826315.242205:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23673 00000100:00100000:3.0:1787826315.242335:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346214528:0@lo:1000:mdt00_012.0 00010000:00010000:3.0:1787826315.242399:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.242409:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.242418:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.242431:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346214656:0@lo:101:mdt00_012.0 00000100:00100000:3.0:1787826315.242457:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.242480:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826315.242589:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346214656 00000100:00100000:1.0:1787826315.242593:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346214656 00000100:00100000:1.0:1787826315.242636:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23674 00000100:00100000:1.0:1787826315.242644:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346214656:12345-0@lo:101:mdt00_012.0 00010000:00010000:1.0:1787826315.242654:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826315.242668:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ab88 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.242692:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.242701:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.242710:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ab88 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.242737:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.242750:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 7 pid: 35098 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.242760:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346214784:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826315.242838:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.242847:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 6 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.242855:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346214912:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826315.242878:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826315.245444:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346215296:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.245517:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.245548:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b618380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346215808:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.245569:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.245586:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346216320:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.245611:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.245648:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1787826315.245654:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346215424:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.245677:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.245694:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b618700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346215936:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.245713:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.245729:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346216448:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.245749:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.245777:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346216320 00000100:00100000:3.0:1787826315.245784:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346216320 00000100:00100000:3.0:1787826315.245832:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346215936 00000100:00100000:3.0:1787826315.245835:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346215936 00000100:00100000:3.0:1787826315.245858:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8815 00000100:00100000:3.0:1787826315.245867:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346216320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.245925:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346216320:12345-0@lo:400:kworker.0 Request processed in 59us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.245932:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8815 00000100:00100000:3.0:1787826315.245949:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8816 00000100:00100000:3.0:1787826315.245954:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346215936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.245975:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346215936:12345-0@lo:400:kworker.0 Request processed in 22us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.245980:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8816 00000100:00100000:3.0:1787826315.245996:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346215296 00000100:00100000:3.0:1787826315.245998:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346215296 00000100:00100000:3.0:1787826315.246021:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346215808 00000100:00100000:3.0:1787826315.246024:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346215808 00000100:00100000:3.0:1787826315.246038:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346215424 00000100:00100000:3.0:1787826315.246040:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346215424 00000100:00100000:3.0:1787826315.246056:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346216448 00000100:00100000:3.0:1787826315.246058:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346216448 00000100:00100000:3.0:1787826315.246075:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14294 00000100:00100000:3.0:1787826315.246081:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346215296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246131:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346215296:12345-0@lo:400:kworker.0 Request processed in 50us (615us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.246138:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14294 00000100:00100000:3.0:1787826315.246230:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14295 00000100:00100000:3.0:1787826315.246237:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346215808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246269:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346215808:12345-0@lo:400:kworker.0 Request processed in 34us (699us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.246276:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14295 00000100:00100000:3.0:1787826315.246287:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14296 00000100:00100000:3.0:1787826315.246293:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346215424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246319:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346215424:12345-0@lo:400:kworker.0 Request processed in 27us (641us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.246325:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14296 00000100:00100000:3.0:1787826315.246337:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14297 00000100:00100000:3.0:1787826315.246343:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346216448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246370:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346216448:12345-0@lo:400:kworker.0 Request processed in 28us (620us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.246376:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14297 00000100:00080000:3.0:1787826315.246422:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826315.246426:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787826315.246429:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826315.246467:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346215424:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246478:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b618700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346215936:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246487:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346216448:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246496:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346215296:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246504:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b618380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346215808:0@lo:400:kworker.0 00000100:00100000:3.0:1787826315.246512:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346216320:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.246650:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346215168:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.246680:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.246701:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346215680:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.246719:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.246733:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346216192:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.246751:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.246767:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1787826315.246773:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346215040:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.246795:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.246818:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346215552:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.246834:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.246847:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346216064:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.246865:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.246886:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346216192 00000100:00100000:2.0:1787826315.246889:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346216192 00000100:00100000:2.0:1787826315.246913:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346215552 00000100:00100000:2.0:1787826315.246916:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346215552 00000100:00100000:2.0:1787826315.246936:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8817 00000100:00100000:2.0:1787826315.246941:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346216192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.246971:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346216192:12345-0@lo:400:kworker.0 Request processed in 30us (219us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.246976:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8817 00000100:00100000:2.0:1787826315.246988:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8818 00000100:00100000:2.0:1787826315.246993:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346215552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247012:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346215552:12345-0@lo:400:kworker.0 Request processed in 20us (178us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.247017:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8818 00000100:00100000:2.0:1787826315.247032:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346215040 00000100:00100000:2.0:1787826315.247034:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346215040 00000100:00100000:2.0:1787826315.247055:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2223 00000100:00100000:2.0:1787826315.247060:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7974f6a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346215040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247084:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7974f6a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346215040:12345-0@lo:400:kworker.0 Request processed in 23us (288us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.247089:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2223 00000100:00080000:2.0:1787826315.247152:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826315.247167:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346215040:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247176:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346215552:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247185:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346216192:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247210:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346216064 00000100:00100000:2.0:1787826315.247212:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346216064 00000100:00100000:2.0:1787826315.247234:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23675 00000100:00100000:2.0:1787826315.247240:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346216064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247275:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346216064:12345-0@lo:400:kworker.0 Request processed in 35us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.247281:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23675 00000100:00100000:2.0:1787826315.247301:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346216064:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247316:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346215168 00000100:00100000:2.0:1787826315.247319:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346215168 00000100:00100000:2.0:1787826315.247336:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346215680 00000100:00100000:2.0:1787826315.247339:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346215680 00000100:00100000:2.0:1787826315.247356:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14298 00000100:00100000:2.0:1787826315.247361:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346215168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247391:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346215168:12345-0@lo:400:kworker.0 Request processed in 29us (710us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.247401:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14298 00000100:00100000:2.0:1787826315.247412:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14299 00000100:00100000:2.0:1787826315.247417:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346215680:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247441:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346215680:12345-0@lo:400:kworker.0 Request processed in 24us (721us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.247447:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14299 00000100:00080000:2.0:1787826315.247461:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826315.247476:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346215168:0@lo:400:kworker.0 00000100:00100000:2.0:1787826315.247486:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346215680:0@lo:400:kworker.0 00000100:00100000:0.0:1787826315.250439:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826315.250529:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826315.250600:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332596864 00000100:00100000:2.0:1787826315.250610:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332596864 00010000:00010000:2.0:1787826315.250670:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 8 pid: 35098 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826315.250714:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332596992 00000100:00100000:2.0:1787826315.250719:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332596992 00010000:00010000:2.0:1787826315.250732:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 7 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826315.250753:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21948 00000100:00100000:2.0:1787826315.250764:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332596864:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826315.250782:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826315.250790:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 8 pid: 35098 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.250805:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 8 pid: 35098 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.250834:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826315.250842:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ab88 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.250853:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826315.250915:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332596864:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 152us (486us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.250923:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21948 00000100:00100000:2.0:1787826315.250943:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21949 00000100:00100000:2.0:1787826315.250949:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332596992:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826315.250958:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826315.250963:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 7 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.250981:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 7 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.250994:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826315.250999:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ab88 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.251009:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4ab88 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.251018:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4ab88 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.251054:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab88 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826315.251070:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346216576:0@lo:105: 00000100:00100000:2.0:1787826315.251153:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826315.251193:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826315.251399:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346216576 00000100:00100000:3.0:1787826315.251405:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346216576 00000100:00100000:3.0:1787826315.251447:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7865 00000100:00100000:3.0:1787826315.251454:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346216576:12345-0@lo:105: 00010000:00010000:3.0:1787826315.251466:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.251476:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.251486:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.251493:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.251500:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.251508:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.251522:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69400) 00000100:00100000:3.0:1787826315.251562:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346216576:12345-0@lo:105: Request processed in 108us (408us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.251569:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7865 00000100:00100000:3.0:1787826315.251612:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346216576:0@lo:105: 00010000:00010000:3.0:1787826315.251641:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826315.251673:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332596992:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 722us (1142us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.251679:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21949 00010000:00010000:1.0:1787826315.257051:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ab81 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014b8 expref: 7 pid: 35098 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.257080:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346214784:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826315.257098:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ab5e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014aa expref: 6 pid: 6529 timeout: 8227 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.257133:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346214912:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826315.257314:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab88 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.257327:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826315.257330:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826315.257334:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826315.257494:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346214656:12345-0@lo:101:mdt00_012.0 Request processed in 14850us (15034us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.257503:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23674 00000100:00100000:3.0:1787826315.262299:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346214656:0@lo:101:mdt00_012.0 00010000:00010000:3.0:1787826315.262332:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826315.262366:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826315.262421:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.262429:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.262439:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.262461:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346216704:0@lo:101:mdt00_012.0 00000100:00100000:3.0:1787826315.262525:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.262559:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826315.262675:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346216704 00000100:00100000:0.0:1787826315.262682:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346216704 00000100:00100000:0.0:1787826315.262740:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23676 00000100:00100000:0.0:1787826315.262749:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346216704:12345-0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826315.262768:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826315.262786:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ab9d lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ab96 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.262812:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ab9d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ab96 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.262826:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ab9d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab96 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.262835:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a000) 00010000:00010000:0.0:1787826315.262838:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826315.262842:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826315.262887:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346216704:12345-0@lo:101:mdt00_012.0 Request processed in 139us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.262907:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23676 00000100:00100000:1.0:1787826315.263333:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346216704:0@lo:101:mdt00_012.0 00010000:00010000:1.0:1787826315.263348:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.263359:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826315.263376:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787826315.263415:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346216832:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826315.263454:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.263480:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.263533:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346216832 00000100:00100000:2.0:1787826315.263539:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346216832 00000100:00100000:2.0:1787826315.263580:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23677 00000100:00100000:2.0:1787826315.263589:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346216832:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826315.263668:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346216832:12345-0@lo:1000:mdt00_012.0 Request processed in 80us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.263675:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23677 00000100:00100000:0.0:1787826315.263830:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346216832:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826315.263881:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade1880 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346216960:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826315.263991:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.264014:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.264065:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346216960 00000100:00100000:2.0:1787826315.264069:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346216960 00000100:00100000:2.0:1787826315.265344:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23678 00000100:00100000:2.0:1787826315.265361:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf89450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346216960:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826315.265502:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf89450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346216960:12345-0@lo:1000:mdt00_012.0 Request processed in 147us (1511us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826315.265510:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23678 00000100:00100000:1.0:1787826315.266088:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade1880 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346216960:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826315.266385:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade1500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217088:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826315.266426:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.266457:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826315.266514:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346217088 00000100:00100000:3.0:1787826315.266520:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346217088 00000100:00100000:3.0:1787826315.266568:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23679 00000100:00100000:3.0:1787826315.266578:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346217088:12345-0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826315.266764:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346217088:12345-0@lo:1000:mdt00_012.0 Request processed in 187us (336us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787826315.266771:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23679 00000100:00100000:0.0:1787826315.266828:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade1500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217088:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826315.266872:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade1500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217216:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826315.266967:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.267035:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.267074:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346217216 00000100:00100000:2.0:1787826315.267079:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346217216 00000100:00100000:2.0:1787826315.270954:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23680 00000100:00100000:2.0:1787826315.270980:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346217216:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826315.271336:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346217216:12345-0@lo:1000:mdt00_012.0 Request processed in 366us (4367us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.271346:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23680 00000100:00100000:1.0:1787826315.271542:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade1500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217216:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826315.271648:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217344:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826315.271693:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.271767:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.271804:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346217344 00000100:00100000:2.0:1787826315.271812:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346217344 00000100:00100000:2.0:1787826315.271856:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23681 00000100:00100000:2.0:1787826315.271865:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346217344:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826315.272042:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346217344:12345-0@lo:1000:mdt00_012.0 Request processed in 178us (348us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.272050:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23681 00000100:00100000:1.0:1787826315.272314:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217344:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826315.272363:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217472:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826315.272389:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.272409:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826315.272456:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346217472 00000100:00100000:0.0:1787826315.272461:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346217472 00000100:00100000:0.0:1787826315.272531:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23682 00000100:00100000:0.0:1787826315.272540:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346217472:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826315.272599:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346217472:12345-0@lo:1000:mdt00_012.0 Request processed in 60us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.272606:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23682 00000100:00100000:1.0:1787826315.272661:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217472:0@lo:1000:mdt00_012.0 00000040:00080000:1.0:1787826315.273594:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1202, 2168 off1095576 00000040:00080000:1.0:1787826315.273687:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1520, 2368 off1537440 00000004:00080000:1.0:1787826315.273693:0:35095:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784c4f4d80 with version 6833 00000004:00080000:3.0:1787826315.273782:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6833 rpc_version 6833 00000020:00100000:3.0:1787826315.273819:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826315.273829:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78508c2a00 x1874663346217728/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826315.273852:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2a00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346217728:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826315.273924:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.274163:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.274405:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346217728 00000100:00100000:2.0:1787826315.274413:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346217728 00000100:00100000:2.0:1787826315.274469:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23683 00000100:00100000:2.0:1787826315.274479:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346217728:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826315.275402:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346217728:12345-0@lo:1000:osp_up0-1.0 Request processed in 921us (1474us total) trans 8589957392 rc 0/0 00000100:00100000:2.0:1787826315.275414:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23683 00000100:00100000:2.0:1787826315.275536:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2a00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346217728:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787826315.275815:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.275832:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.275844:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.275864:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.275871:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.275877:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.275921:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.275928:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.275937:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.275957:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217856:0@lo:101:mdt00_012.0 00000100:00100000:3.0:1787826315.275991:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.276020:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aac00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826315.276151:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346217856 00000100:00100000:0.0:1787826315.276158:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346217856 00000100:00100000:0.0:1787826315.276203:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23684 00000100:00100000:0.0:1787826315.276213:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:35095:x1874663346217856:12345-0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826315.276228:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826315.276244:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4abb9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4abb2 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.276264:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4abb9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4abb2 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.276277:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4abb9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4abb2 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.276285:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826315.276288:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826315.276291:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826315.276334:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346217856:12345-0@lo:101:mdt00_012.0 Request processed in 123us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.276343:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23684 00000100:00100000:3.0:1787826315.276529:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346217856:0@lo:101:mdt00_012.0 00010000:00010000:3.0:1787826315.276544:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276555:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826315.276577:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000009878b59 reply state 00000000c262ea3e transno 4294985402 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276590:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276597:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:3.0:1787826315.276599:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826315.276603:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826315.276609:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276618:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826315.276826:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x193:0x0] 00000004:00010000:3.0:1787826315.276835:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000009878b59 reply state 00000000c262ea3e transno 4294985402 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276851:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276857:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:3.0:1787826315.276861:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826315.276863:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787826315.276868:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4e:0x0]: returning size 4096 00000004:00080000:3.0:1787826315.276877:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826315.276898:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276907:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276914:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276943:0:35095:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826315.276947:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826315.276952:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.276959:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.277046:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224945:x1874663332596736:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 36025us (36228us total) trans 4294985402 rc 0/0 00000100:00100000:3.0:1787826315.277055:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68431 00010000:00010000:2.0:1787826315.277277:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826315.277284:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.277293:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826315.277299:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826315.277308:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.277336:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.277343:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826315.277374:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.277382:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4ab88 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ab8f expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.277394:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.277395:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.277397:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826315.277404:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826315.277425:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346217984:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826315.277465:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.277871:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346217984 00000100:00100000:0.0:1787826315.277878:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346217984 00000100:00100000:0.0:1787826315.277933:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11096 00000100:00100000:0.0:1787826315.277943:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346217984:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826315.277954:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826315.277973:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4ab8f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ab88 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.277987:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826315.277991:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826315.278037:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346217984:12345-0@lo:103:ldlm_bl.0 Request processed in 95us (571us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.278043:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11096 00000100:00100000:3.0:1787826315.278312:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346217984:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826315.284603:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826315.284745:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332597120 00000100:00100000:1.0:1787826315.284756:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332597120 00000100:00100000:1.0:1787826315.284822:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68432 00000100:00100000:1.0:1787826315.284833:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224934:x1874663332597120:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826315.284853:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826315.284874:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4abc0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014bf expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.284956:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4abc7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.284971:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4abc7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.284982:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4abc7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826315.285034:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826315.285038:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826315.285058:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4abc7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.285084:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4abc0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014bf expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.285389:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4abc7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014bf expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.285400:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826315.285403:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826315.285407:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826315.285581:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224934:x1874663332597120:12345-192.168.203.56@tcp:101:bash.0 Request processed in 749us (987us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.285590:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68432 00000100:00100000:0.0:1787826315.292212:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.292406:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332597248 00000100:00100000:3.0:1787826315.292485:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332597248 00000100:00100000:3.0:1787826315.292554:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68433 00000100:00100000:3.0:1787826315.292564:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224934:x1874663332597248:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826315.292585:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826315.292607:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4abce lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014c6 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.292680:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.292692:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.292702:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.292712:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.292721:0:24702:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ab9d lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ab96 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.292729:0:24702:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.292760:0:24702:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ab9d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4ab96 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.292780:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:24702:1874663346218112:0@lo:104: 00000100:00100000:3.0:1787826315.292835:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.292878:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826315.293039:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346218112 00000100:00100000:0.0:1787826315.293043:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346218112 00000100:00100000:0.0:1787826315.293079:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7866 00000100:00100000:0.0:1787826315.293086:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:24702:x1874663346218112:12345-0@lo:104: 00010000:00010000:0.0:1787826315.293098:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.293430:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:24702:1874663346218112:0@lo:104: 00010000:00010000:1.0:1787826315.293449:0:24702:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.293536:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826315.293544:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:24702:x1874663346218112:12345-0@lo:104: Request processed in 458us (707us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.293551:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7866 00010000:00010000:2.0:1787826315.293771:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826315.293779:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.293787:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826315.293793:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826315.293801:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.293828:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.293834:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826315.293883:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346218240:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826315.293935:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.294055:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346218240 00000100:00100000:0.0:1787826315.294062:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346218240 00010000:00010000:0.0:1787826315.294082:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ab9d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4ab96 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826315.294150:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21950 00000100:00100000:0.0:1787826315.294158:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd93050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346218240:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826315.294168:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826315.294563:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.294573:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69800/0xe21309aa81f4ab96 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ab9d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.294595:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.294597:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826315.294599:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787826315.298174:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957392 is committed 00010000:00010000:0.0:1787826315.298536:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ab9d lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4ab96 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.298560:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a000) 00010000:00010000:0.0:1787826315.298566:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.298576:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.298583:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.298631:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826315.298734:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd93050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346218240:12345-0@lo:103:ldlm_bl.0 Request processed in 4569us (4790us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.298745:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21950 00010000:00010000:1.0:1787826315.298877:0:24702:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.298886:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826315.299029:0:24702:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:24702:1874663346218368:0@lo:1000:mdt00_006.0 00000100:00100000:1.0:1787826315.299070:0:24702:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.299101:0:24702:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.299251:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346218368 00000100:00100000:2.0:1787826315.299264:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346218368 00000100:00100000:2.0:1787826315.299355:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23685 00000100:00100000:2.0:1787826315.299368:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:24702:x1874663346218368:12345-0@lo:1000:mdt00_006.0 00000100:00100000:2.0:1787826315.299507:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:24702:x1874663346218368:12345-0@lo:1000:mdt00_006.0 Request processed in 143us (436us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.299516:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23685 00000100:00100000:3.0:1787826315.299925:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346218240:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826315.300020:0:24702:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-mdtlov_UUID:24702:1874663346218368:0@lo:1000:mdt00_006.0 00010000:00010000:1.0:1787826315.300078:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4abe3 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300094:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4abe3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300272:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4abe3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826315.300301:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4abe3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300319:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300325:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300332:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300339:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300345:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826315.300349:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300357:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300369:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826315.300372:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826315.300376:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300383:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4abd5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300397:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300404:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300482:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300490:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826315.300494:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300500:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300509:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a000) 00010000:00010000:1.0:1787826315.300511:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826315.300514:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300520:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4abdc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300562:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4abce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014c6 expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300581:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4abe3 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014c6 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.300588:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f794385a400) 00010000:00010000:1.0:1787826315.300591:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826315.300593:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826315.300673:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224934:x1874663332597248:12345-192.168.203.56@tcp:101:bash.0 Request processed in 8110us (8470us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.300682:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68433 00000100:00100000:0.0:1787826315.303327:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.303517:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332597376 00000100:00100000:3.0:1787826315.303528:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332597376 00000100:00100000:3.0:1787826315.303601:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68434 00000100:00100000:3.0:1787826315.303613:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224934:x1874663332597376:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826315.303649:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826315.303673:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4abea lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014cd expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.303742:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4abf1 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.303756:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4abf1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.303767:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4abf1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826315.303869:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4e:0x0]: returning size 4096 00000004:00200000:3.0:1787826315.303875:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826315.303914:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4abea lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014cd expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.303941:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4abf1 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014cd expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.303950:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:3.0:1787826315.303953:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826315.303957:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:3.0:1787826315.304029:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224934:x1874663332597376:12345-192.168.203.56@tcp:101:bash.0 Request processed in 418us (711us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.304037:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68434 00000100:00100000:2.0:1787826315.306880:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826315.307163:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332597504 00000100:00100000:0.0:1787826315.307174:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332597504 00000100:00100000:0.0:1787826315.307242:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68435 00000100:00100000:0.0:1787826315.307253:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224934:x1874663332597504:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826315.307277:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826315.307300:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4abf8 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014d4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.307373:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4abff lrc: 3/1,0 mode: --/PR res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.307387:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4abff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.307397:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4abff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826315.307447:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x195:0x0]: returning size 4096 00000004:00200000:0.0:1787826315.307453:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826315.307493:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4abf8 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014d4 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.307512:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4abff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014d4 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.307522:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:0.0:1787826315.307525:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826315.307529:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826315.307601:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224934:x1874663332597504:12345-192.168.203.56@tcp:101:bash.0 Request processed in 349us (730us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.307608:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68435 00000100:00100000:2.0:1787826315.315826:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826315.315977:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332597632 00000100:00100000:0.0:1787826315.315989:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332597632 00000100:00100000:0.0:1787826315.316058:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68436 00000100:00100000:0.0:1787826315.316069:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:224934:x1874663332597632:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826315.316093:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826315.316140:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ac06 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014db expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.316208:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.316223:0:35098:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4abb9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4abb2 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.316233:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 4/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.316268:0:35098:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4abb9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4abb2 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826315.316292:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade2680 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000_UUID:35098:1874663346218496:0@lo:104: 00000100:00100000:0.0:1787826315.316340:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826315.316380:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d79c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.316508:0:6509:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787826315.316520:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346218496 00000100:00100000:2.0:1787826315.316523:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346218496 00000100:00100000:2.0:1787826315.316559:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7867 00000100:00100000:2.0:1787826315.316566:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7946048850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35098:x1874663346218496:12345-0@lo:104: 00010000:00010000:2.0:1787826315.316578:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.316657:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826315.316663:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7946048850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35098:x1874663346218496:12345-0@lo:104: Request processed in 98us (322us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.316670:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7867 00000100:00100000:1.0:1787826315.316691:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade2680 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000_UUID:35098:1874663346218496:0@lo:104: 00010000:00010000:1.0:1787826315.316710:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.316758:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826315.316765:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.316773:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826315.316778:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826315.316788:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.316815:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.316821:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826315.316851:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.316863:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4abb2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4abb9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.316880:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826315.316881:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826315.316883:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826315.316996:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826315.317011:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346218624:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826315.317046:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.317751:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346218624 00000100:00100000:3.0:1787826315.317759:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346218624 00010000:00010000:3.0:1787826315.317787:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4abb9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4abb2 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.317873:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21951 00000100:00100000:3.0:1787826315.317882:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346218624:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826315.317903:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826315.318676:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4abb9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4abb2 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.318694:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:3.0:1787826315.318699:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.318707:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.318714:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.318746:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826315.318806:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346218624:12345-0@lo:103:ldlm_bl.0 Request processed in 924us (1758us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826315.318814:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21951 00000100:00100000:1.0:1787826315.318912:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346218624:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826315.318939:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.318947:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826315.319007:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x193:0x0]: returning size 4096 00000004:00200000:1.0:1787826315.319012:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826315.319058:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ac06 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014db expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.319072:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014db expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.319079:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:1.0:1787826315.319082:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826315.319085:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:1.0:1787826315.319177:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224934:x1874663332597632:12345-192.168.203.56@tcp:101:bash.0 Request processed in 3110us (3362us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.319185:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68436 00000100:00100000:0.0:1787826315.326780:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826315.326882:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332597888 00000100:00100000:1.0:1787826315.326893:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332597888 00000100:00100000:1.0:1787826315.326961:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68437 00000100:00100000:1.0:1787826315.326973:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:224934:x1874663332597888:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826315.326995:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826315.326998:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787826315.327027:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac0d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014db expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327047:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:1.0:1787826315.327059:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826315.327076:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4ac14 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014e2 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327246:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327264:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327273:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327286:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327294:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327300:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826315.327637:0:6528:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x520:0x0], index 1: 0 00010000:00010000:1.0:1787826315.327773:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ac29 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327783:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ac29 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327790:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ac29 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327823:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327830:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327836:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327843:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327849:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826315.327853:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327861:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327871:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794385a000) 00010000:00010000:1.0:1787826315.327874:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826315.327878:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 3/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327885:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac1b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826315.327897:0:6528:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fcf6e541 reply state 000000003e70eb1c transno 8589957393 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327907:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327912:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794385a500) 00010000:00010000:1.0:1787826315.327915:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826315.327917:0:6528:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826315.327947:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4ac14 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014e2 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327970:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ac29 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014e2 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.327977:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f794385a800) 00010000:00010000:1.0:1787826315.327979:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826315.327981:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826315.328042:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224934:x1874663332597888:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1073us (1273us total) trans 8589957393 rc 0/0 00000100:00100000:1.0:1787826315.328049:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68437 00000100:00100000:0.0:1787826315.332051:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826315.332545:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332598016 00000100:00100000:2.0:1787826315.332554:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332598016 00000100:00100000:2.0:1787826315.332732:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30827 00000100:00100000:2.0:1787826315.332744:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224934:x1874663332598016:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826315.332763:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826315.332786:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014e9 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826315.332812:0:8445:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x520:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826315.332837:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=8127s v=(0 0) 00010000:00010000:2.0:1787826315.332842:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826315.332852:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 7 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.332870:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 7 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.332883:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826315.332953:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:224934:x1874663332598016:12345-192.168.203.56@tcp:101:bash.0 Request processed in 212us (912us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.332960:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30827 00000100:00100000:2.0:1787826315.337269:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.337572:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332598272 00000100:00100000:3.0:1787826315.337583:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332598272 00000100:00100000:3.0:1787826315.337685:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9345 00000100:00100000:3.0:1787826315.337697:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:224934:x1874663332598272:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826315.338012:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd1850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224934:x1874663332598272:12345-192.168.203.56@tcp:35:bash.0 Request processed in 317us (903us total) trans 8589957394 rc 0/0 00000100:00100000:3.0:1787826315.338021:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9345 00000100:00100000:0.0:1787826315.341673:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.341759:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332598528 00000100:00100000:3.0:1787826315.341770:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332598528 00000100:00100000:3.0:1787826315.341885:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68438 00000100:00100000:3.0:1787826315.341905:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:224934:x1874663332598528:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826315.341923:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826315.341926:0:11629:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826315.341952:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4abff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014d4 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.341970:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:3.0:1787826315.341974:0:11629:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826315.341986:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4ac3e lrc: 2/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014f0 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.342156:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.342169:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.342179:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.342194:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.342200:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.342206:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826315.342339:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346218752:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787826315.342396:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826315.342434:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826315.343405:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346218752 00000100:00100000:2.0:1787826315.343413:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346218752 00000100:00100000:2.0:1787826315.343465:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23686 00000100:00100000:2.0:1787826315.343474:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11629:x1874663346218752:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787826315.344060:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11629:x1874663346218752:12345-0@lo:1000:mdt00_004.0 Request processed in 585us (1661us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.344068:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23686 00000100:00100000:3.0:1787826315.344364:0:11629:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957392 gen 2 00000004:00080000:3.0:1787826315.344370:0:11629:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957392 committed transno 8589957392 00000100:00100000:3.0:1787826315.344405:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11629:1874663346218752:0@lo:1000:mdt00_004.0 00000004:00080000:3.0:1787826315.345441:0:11629:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4cd:0x0], index 0: 0 00010000:00010000:3.0:1787826315.347472:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4ac53 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347652:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4ac53 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347666:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4ac53 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347713:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347721:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347727:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347735:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347740:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826315.347745:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347753:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347766:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:3.0:1787826315.347769:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826315.347773:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347779:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac45 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826315.347793:0:11629:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000037d0b48b reply state 000000009243f06a transno 4294985403 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347805:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347811:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x4ec94fd6 (ffff9f7843326500) 00010000:00010000:3.0:1787826315.347814:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826315.347817:0:11629:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826315.347855:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4ac3e lrc: 0/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014f0 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347869:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4ac53 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014f0 expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826315.347876:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7843326100) 00010000:00010000:3.0:1787826315.347878:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826315.347881:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826315.347968:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224934:x1874663332598528:12345-192.168.203.56@tcp:101:bash.0 Request processed in 6064us (6304us total) trans 4294985403 rc 0/0 00000100:00100000:3.0:1787826315.347977:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68438 00000100:00100000:3.0:1787826315.354540:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826315.356706:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332598656 00000100:00100000:2.0:1787826315.356722:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332598656 00000100:00100000:2.0:1787826315.356818:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30828 00000100:00100000:2.0:1787826315.356831:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:224934:x1874663332598656:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826315.356850:0:20167:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826315.356875:0:20167:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014f7 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826315.356901:0:20167:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4cd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826315.356924:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cd:0x0].0x0 since=8127s v=(0 0) 00010000:00010000:2.0:1787826315.356929:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826315.356938:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826315.356956:0:20167:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826315.356969:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787826315.356971:0:20167:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826315.357049:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:224934:x1874663332598656:12345-192.168.203.56@tcp:101:bash.0 Request processed in 220us (2517us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826315.357056:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30828 00000100:00100000:0.0:1787826315.362688:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826315.362754:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332598912 00000100:00100000:3.0:1787826315.362764:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332598912 00000100:00100000:3.0:1787826315.362830:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9346 00000100:00100000:3.0:1787826315.362840:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:224934:x1874663332598912:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826315.363207:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:224934:x1874663332598912:12345-192.168.203.56@tcp:35:bash.0 Request processed in 367us (528us total) trans 4294985404 rc 0/0 00000100:00100000:3.0:1787826315.363216:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9346 00000100:00100000:2.0:1787826315.376669:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826315.376774:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826315.376909:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826315.376996:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332599040 00000100:00100000:0.0:1787826315.377006:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332599040 00000100:00100000:0.0:1787826315.377076:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332599168 00000100:00100000:0.0:1787826315.377079:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332599168 00000100:00100000:0.0:1787826315.377144:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11097 00000100:00100000:0.0:1787826315.377155:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:6167:x1874663332599040:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826315.377170:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826315.377198:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4abc7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014bf expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.377222:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826315.377234:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ac29 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014e2 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.377245:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f794385a800) 00010000:00010000:0.0:1787826315.377259:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4abe3 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014c6 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.377269:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f794385a400) 00010000:00010000:0.0:1787826315.377274:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826315.377334:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:6167:x1874663332599040:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 181us (677us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.377341:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11097 00000100:00100000:0.0:1787826315.377360:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11098 00000100:00100000:0.0:1787826315.377367:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:34879:x1874663332599168:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826315.377376:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826315.377387:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4aa46 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dba:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801472 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826315.377398:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dba:0x0].0x0 (ffff9f79460a2d00) 00010000:00010000:0.0:1787826315.377406:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826315.377434:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:34879:x1874663332599168:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 67us (659us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826315.377440:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11098 00000100:00100000:1.0:1787826315.377850:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332599296 00000100:00100000:1.0:1787826315.377857:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332599296 00000100:00100000:1.0:1787826315.377902:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11099 00000100:00100000:1.0:1787826315.377910:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:15672:x1874663332599296:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826315.377920:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826315.377935:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4ac53 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014f0 expref: 13 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.377948:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7843326100) 00010000:00010000:1.0:1787826315.377962:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4abf1 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014cd expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.377971:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:1.0:1787826315.377980:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4a9ac lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080144f expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826315.377995:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4c:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:1.0:1787826315.378001:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826315.378033:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:15672:x1874663332599296:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 123us (1123us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.378039:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11099 00010000:00010000:2.0:1787826315.436468:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826315.756855:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826315.756883:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346218880:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826315.757000:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826315.757484:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346218880 00000100:00100000:1.0:1787826315.757493:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346218880 00000100:00100000:1.0:1787826315.757552:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6227 00000100:00100000:1.0:1787826315.757562:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346218880:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826315.757593:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914711040 avail 3683188736 00000100:00100000:1.0:1787826315.757669:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346218880:12345-0@lo:13:osp-pre-1-1.0 Request processed in 109us (669us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826315.757677:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6227 00000100:00100000:0.0:1787826315.757815:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346218880:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826316.268286:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346219008:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826316.268377:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826316.268790:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346219008 00000100:00100000:3.0:1787826316.268802:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346219008 00000100:00100000:3.0:1787826316.268867:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23687 00000100:00100000:3.0:1787826316.268879:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346219008:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826316.268912:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346219136:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826316.268953:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826316.269000:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346219136 00000100:00100000:1.0:1787826316.269004:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346219136 00000100:00100000:1.0:1787826316.269061:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6228 00000100:00100000:1.0:1787826316.269078:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346219136:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826316.269140:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826316.269196:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346219136:12345-0@lo:13:osp-pre-0-0.0 Request processed in 128us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826316.269204:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6228 00000100:00080000:1.0:1787826316.269232:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826316.269905:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346219264:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826316.269945:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826316.269974:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826316.269981:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845aff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346219392:0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787826316.270065:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443405824 avail 1297166336 00000100:00100000:3.0:1787826316.270128:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346219008:12345-0@lo:41:osp-pre-1-0.0 Request processed in 1250us (1750us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826316.270136:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23687 00000100:00100000:1.0:1787826316.270405:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826316.270444:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346219264 00000100:00100000:1.0:1787826316.270451:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346219264 00000100:00100000:1.0:1787826316.270494:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6229 00000100:00100000:1.0:1787826316.270507:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346219264:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826316.270525:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914711040 avail 3683188736 00000100:00100000:1.0:1787826316.270562:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346219264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 59us (617us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826316.270568:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6229 00000100:00100000:1.0:1787826316.270591:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845aff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346219520:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826316.270630:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826316.270719:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346219008:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826316.270724:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346219520 00000100:00100000:0.0:1787826316.270729:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346219520 00000100:00100000:0.0:1787826316.270759:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6230 00000100:00100000:0.0:1787826316.270765:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346219520:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826316.270770:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346219136:0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787826316.270776:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826316.270799:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346219264:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826316.270815:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346219520:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826316.270822:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6230 00000100:00100000:1.0:1787826316.270847:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845aff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346219520:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826316.272032:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346219392 00000100:00100000:3.0:1787826316.272048:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346219392 00000100:00100000:3.0:1787826316.272124:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23688 00000100:00100000:3.0:1787826316.272136:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346219392:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787826316.272173:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442795520 avail 1296556032 00000100:00100000:3.0:1787826316.272249:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346219392:12345-0@lo:41:osp-pre-0-1.0 Request processed in 116us (1843us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826316.272257:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23688 00000100:00100000:1.0:1787826316.272484:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845aff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346219392:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1787826316.461437:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826316.620083:0:166052:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:3.0:1787826316.620729:0:166052:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:2.0:1787826317.484297:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826318.508743:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826319.297629:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826319.297704:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332599424 00000100:00100000:1.0:1787826319.297715:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332599424 00000100:00100000:1.0:1787826319.297799:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68439 00000100:00100000:1.0:1787826319.297811:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225115:x1874663332599424:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826319.297831:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826319.297853:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac68 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608014fe expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.297924:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.297936:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.297949:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826319.298005:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826319.298010:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826319.298033:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.298060:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ac68 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.298076:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.298083:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826319.298086:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826319.298090:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826319.298174:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225115:x1874663332599424:12345-192.168.203.56@tcp:101:cat.0 Request processed in 367us (556us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.298182:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68439 00000100:00080000:3.0:1787826319.298444:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826319.304514:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826319.304560:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332599552 00000100:00100000:0.0:1787826319.304569:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332599552 00000100:00100000:0.0:1787826319.304638:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68440 00000100:00100000:0.0:1787826319.304649:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225115:x1874663332599552:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826319.304671:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826319.304694:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ac76 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801505 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304762:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304776:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304788:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304803:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304810:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304816:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304888:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac8b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304897:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac8b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304903:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac8b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826319.304926:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac8b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304940:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304947:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304953:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304960:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304965:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826319.304981:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.304989:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305001:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826319.305005:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826319.305009:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305016:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ac7d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305030:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305035:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305041:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305046:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826319.305049:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305054:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305061:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5300) 00010000:00010000:0.0:1787826319.305063:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826319.305066:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305072:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ac84 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305102:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ac76 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801505 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305134:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac8b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801505 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.305141:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826319.305144:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826319.305146:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826319.305205:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225115:x1874663332599552:12345-192.168.203.56@tcp:101:cat.0 Request processed in 558us (700us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.305213:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68440 00000100:00100000:1.0:1787826319.317667:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.317762:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332599680 00000100:00100000:3.0:1787826319.317774:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332599680 00000100:00100000:3.0:1787826319.317850:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68441 00000100:00100000:3.0:1787826319.317862:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225115:x1874663332599680:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826319.317886:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826319.317909:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac92 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080150c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.317971:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4ac99 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.317986:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4ac99 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.317999:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4ac99 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826319.318100:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4e:0x0]: returning size 4096 00000004:00200000:3.0:1787826319.318130:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826319.318170:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ac92 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080150c expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.318186:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4ac99 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080150c expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.318195:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:3.0:1787826319.318198:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826319.318202:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826319.318263:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225115:x1874663332599680:12345-192.168.203.56@tcp:101:cat.0 Request processed in 404us (607us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.318271:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68441 00000100:00080000:1.0:1787826319.318505:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826319.321527:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826319.321615:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332599808 00000100:00100000:2.0:1787826319.321625:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332599808 00000100:00100000:2.0:1787826319.321688:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68442 00000100:00100000:2.0:1787826319.321699:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225115:x1874663332599808:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826319.321719:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826319.321742:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4aca0 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801513 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.321803:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.321817:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.321828:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826319.321876:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x195:0x0]: returning size 4096 00000004:00200000:2.0:1787826319.321881:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826319.321914:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4aca0 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 14 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.321930:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 13 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.321938:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:2.0:1787826319.321940:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826319.321944:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826319.322006:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225115:x1874663332599808:12345-192.168.203.56@tcp:101:cat.0 Request processed in 308us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.322013:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68442 00000100:00100000:3.0:1787826319.331394:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826319.331542:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332599936 00000100:00100000:1.0:1787826319.331553:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332599936 00000100:00100000:1.0:1787826319.331632:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68443 00000100:00100000:1.0:1787826319.331644:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225115:x1874663332599936:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826319.331668:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826319.331696:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4acae lrc: 2/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080151a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.331763:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.331778:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.331789:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826319.331839:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x193:0x0]: returning size 4096 00000004:00200000:1.0:1787826319.331845:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826319.331884:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4acae lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.331900:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 9 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.331909:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794385aa00) 00010000:00010000:1.0:1787826319.331912:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826319.331916:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826319.331983:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225115:x1874663332599936:12345-192.168.203.56@tcp:101:cat.0 Request processed in 340us (596us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.331990:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68443 00000100:00100000:1.0:1787826319.337343:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826319.337394:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332600192 00000100:00100000:1.0:1787826319.337404:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332600192 00000100:00100000:1.0:1787826319.337462:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68444 00000100:00100000:1.0:1787826319.337473:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225115:x1874663332600192:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826319.337494:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826319.337516:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4acbc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801521 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.337604:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.337618:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.337629:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.337640:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 3/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.337646:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.337652:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:1.0:1787826319.337718:0:6528:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00010000:00010000:2.0:1787826319.533480:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826319.580637:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.580689:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.580722:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.580755:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.580781:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.580801:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.580824:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.580854:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332600448 00000100:00100000:3.0:1787826319.580871:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332600448 00000100:00100000:3.0:1787826319.580934:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332601088 00000100:00100000:3.0:1787826319.580938:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332601088 00000100:00100000:3.0:1787826319.580960:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332600960 00000100:00100000:3.0:1787826319.580963:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332600960 00000100:00100000:3.0:1787826319.580985:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332600576 00000100:00100000:3.0:1787826319.580989:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332600576 00000100:00100000:3.0:1787826319.581033:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8819 00000100:00100000:3.0:1787826319.581044:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332600448:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826319.581150:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332600448:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 107us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.581157:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8819 00000100:00100000:3.0:1787826319.581175:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8820 00000100:00100000:3.0:1787826319.581182:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332601088:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826319.581208:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332601088:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (486us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.581214:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8820 00000100:00100000:3.0:1787826319.581224:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8821 00000100:00100000:3.0:1787826319.581230:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332600960:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826319.581249:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332600960:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 19us (448us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.581254:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8821 00000100:00100000:3.0:1787826319.581263:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30829 00000100:00100000:3.0:1787826319.581268:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2392:x1874663332600576:12345-192.168.203.56@tcp:9:kworker.0 00000020:00000080:3.0:1787826319.581286:0:8443:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xe21309aa81f49944 00000100:00080000:3.0:1787826319.581294:0:8443:0:(import.c:65:import_set_state_nolock()) ffff9f784407b800 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1787826319.581341:0:8443:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294976156 00000100:00100000:3.0:1787826319.581460:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+2:2392:x1874663332600576:12345-192.168.203.56@tcp:9:kworker.0 Request processed in 189us (634us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.581467:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30829 00000100:00100000:3.0:1787826319.581488:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332600832 00000100:00100000:3.0:1787826319.581492:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332600832 00000100:00100000:3.0:1787826319.581517:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332600704 00000100:00100000:3.0:1787826319.581520:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332600704 00000100:00100000:3.0:1787826319.581539:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14300 00000100:00100000:3.0:1787826319.581546:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2396:x1874663332600832:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826319.581576:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2396:x1874663332600832:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (887us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.581582:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14300 00000100:00100000:3.0:1787826319.581607:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14301 00000100:00100000:3.0:1787826319.581614:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332600704:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826319.581640:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332600704:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (858us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.581646:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14301 00010000:00010000:3.0:1787826319.581788:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826319.582206:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332600320 00000100:00100000:0.0:1787826319.582212:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332600320 00000100:00100000:2.0:1787826319.582268:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2224 00000100:00100000:2.0:1787826319.582277:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943ed5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332600320:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826319.582324:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943ed5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332600320:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 48us (1568us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.582331:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2224 00000100:00080000:1.0:1787826319.582451:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787826319.582456:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826319.582458:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826319.582461:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826319.807247:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826319.807362:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332601216 00000100:00100000:0.0:1787826319.807388:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332601216 00000100:00100000:0.0:1787826319.807458:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68445 00000100:00100000:0.0:1787826319.807470:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225117:x1874663332601216:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826319.807492:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826319.807515:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4acd1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801528 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.807603:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.807616:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.807630:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826319.807687:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826319.807690:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826319.807713:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.807739:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4acd1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.807758:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.807767:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826319.807771:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826319.807774:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826319.807844:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225117:x1874663332601216:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 376us (607us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.807851:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68445 00000100:00100000:2.0:1787826319.820635:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.820802:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332601344 00000100:00100000:3.0:1787826319.820812:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332601344 00000100:00100000:3.0:1787826319.820985:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68446 00000100:00100000:3.0:1787826319.821007:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225117:x1874663332601344:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826319.821029:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826319.821053:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4acdf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080152f expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821160:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821174:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821185:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821202:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821208:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821214:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821286:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821295:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821301:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826319.821324:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821338:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821345:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821352:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821358:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821364:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826319.821384:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821393:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821405:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826319.821408:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826319.821413:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821418:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ace6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821433:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821439:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821444:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821450:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826319.821453:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821459:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821466:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7843326700) 00010000:00010000:3.0:1787826319.821468:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826319.821470:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821476:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4aced lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821508:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4acdf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821521:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.821527:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:3.0:1787826319.821529:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826319.821532:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826319.822050:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225117:x1874663332601344:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1053us (1423us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.822057:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68446 00000100:00100000:1.0:1787826319.825855:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826319.826094:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332601472 00000100:00100000:2.0:1787826319.826219:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332601472 00000100:00100000:2.0:1787826319.826309:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68447 00000100:00100000:2.0:1787826319.826320:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225117:x1874663332601472:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826319.826340:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826319.826362:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4acfb lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801536 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.826436:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4ad02 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.826449:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4ad02 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.826462:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4ad02 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826319.826563:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4e:0x0]: returning size 4096 00000004:00200000:2.0:1787826319.826568:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826319.826619:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4acfb lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801536 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.826637:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4ad02 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801536 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.826645:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:2.0:1787826319.826648:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826319.826652:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826319.826715:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225117:x1874663332601472:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 397us (870us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.826723:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68447 00000100:00100000:1.0:1787826319.834975:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826319.835355:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332601600 00000100:00100000:2.0:1787826319.835377:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332601600 00000100:00100000:2.0:1787826319.835440:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68448 00000100:00100000:2.0:1787826319.835452:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225117:x1874663332601600:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826319.835475:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826319.835498:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4ad09 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080153d expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835566:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835581:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835602:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835615:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835622:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835629:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835729:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ad1e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835737:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ad1e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835744:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ad1e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826319.835789:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4f:0x0]: returning size 6 00000004:00200000:2.0:1787826319.835793:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826319.835813:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ad1e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835824:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ad1e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835834:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835841:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835847:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.835853:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836095:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826319.836101:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836186:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836200:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:2.0:1787826319.836203:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826319.836208:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836214:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4ad10 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836229:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836235:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836241:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836246:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826319.836249:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836256:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836263:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x4ec94fd6 (ffff9f7843326500) 00010000:00010000:2.0:1787826319.836265:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826319.836268:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836274:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4ad17 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836305:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4ad09 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080153d expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836320:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ad1e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080153d expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.836327:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:2.0:1787826319.836329:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826319.836332:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826319.836579:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225117:x1874663332601600:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1127us (1613us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.836586:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68448 00000100:00100000:1.0:1787826319.841953:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826319.843449:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332601728 00000100:00100000:0.0:1787826319.843465:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332601728 00000100:00100000:0.0:1787826319.843620:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68449 00000100:00100000:0.0:1787826319.843633:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225117:x1874663332601728:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826319.843655:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826319.843680:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ad25 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801544 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843752:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843766:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843777:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843788:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843796:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843801:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843912:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ad3a lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843920:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ad3a lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.843927:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ad3a lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826319.843989:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbb:0x0]: returning size 6 00000004:00200000:0.0:1787826319.843994:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826319.844016:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ad3a lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844027:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ad3a lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844036:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844042:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844047:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844054:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844059:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826319.844063:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844070:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844082:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794385aa00) 00010000:00010000:0.0:1787826319.844084:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826319.844088:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 3/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844094:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ad2c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844145:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844152:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844158:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844163:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826319.844165:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844171:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844179:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794385a500) 00010000:00010000:0.0:1787826319.844181:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826319.844184:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844190:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad33 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844216:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ad25 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801544 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844243:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ad3a lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801544 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.844251:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f784b4a5300) 00010000:00010000:0.0:1787826319.844253:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826319.844255:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826319.844329:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225117:x1874663332601728:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 697us (2386us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.844336:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68449 00000100:00100000:3.0:1787826319.847768:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.847815:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826319.847938:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332601856 00000100:00100000:2.0:1787826319.847959:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332601856 00000100:00100000:2.0:1787826319.848039:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11100 00000100:00100000:2.0:1787826319.848050:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:2393:x1874663332601856:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:2.0:1787826319.848065:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826319.848092:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ad1e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080153d expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.848133:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:2.0:1787826319.848144:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826319.848199:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2393:x1874663332601856:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 151us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.848208:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11100 00000100:00100000:0.0:1787826319.848778:0:6541:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332602112 00000100:00100000:0.0:1787826319.848785:0:6541:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332602112 00000100:00100000:0.0:1787826319.848834:0:6541:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1230 00000100:00100000:0.0:1787826319.848843:0:6541:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b619880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225117:x1874663332602112:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:0.0:1787826319.848902:0:6541:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826319.848920:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ad3a lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801544 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.848933:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f784b4a5300) 00010000:00010000:0.0:1787826319.848952:0:6541:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826319.849035:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346219648:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826319.849089:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826319.849169:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.849325:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346219648 00000100:00100000:3.0:1787826319.849330:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346219648 00000100:00100000:3.0:1787826319.849368:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23689 00000100:00100000:3.0:1787826319.849376:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346219648:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.849455:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346219648:12345-0@lo:1000:mdt_io00_001.0 Request processed in 80us (365us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.849462:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23689 00000100:00100000:1.0:1787826319.849519:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346219648:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.849587:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346219776:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.849631:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.849670:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.849724:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346219776 00000100:00100000:3.0:1787826319.849729:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346219776 00000100:00100000:3.0:1787826319.849760:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23690 00000100:00100000:3.0:1787826319.849768:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346219776:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.849839:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346219776:12345-0@lo:1000:mdt_io00_001.0 Request processed in 72us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.849846:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23690 00000100:00100000:1.0:1787826319.849953:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346219776:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826319.849982:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x195:0x0] 00000100:00100000:1.0:1787826319.850020:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346219904:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.850043:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.850065:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.850278:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346219904 00000100:00100000:3.0:1787826319.850282:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346219904 00000100:00100000:3.0:1787826319.850313:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23691 00000100:00100000:3.0:1787826319.850320:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346219904:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.850747:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346219904:12345-0@lo:1000:mdt_io00_001.0 Request processed in 425us (700us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.850755:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23691 00000100:00100000:1.0:1787826319.850831:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346219904:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826319.850854:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4e:0x0] 00000100:00100000:1.0:1787826319.850885:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346220032:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.850911:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.851035:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.851328:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346220032 00000100:00100000:2.0:1787826319.851335:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346220032 00000100:00100000:3.0:1787826319.851451:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23692 00000100:00100000:3.0:1787826319.851462:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346220032:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.851610:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346220032:12345-0@lo:1000:mdt_io00_001.0 Request processed in 149us (696us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.851619:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23692 00000100:00100000:1.0:1787826319.851725:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346220032:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826319.851766:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x195:0x0] 00000100:00100000:1.0:1787826319.851792:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346220160:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.851820:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.852548:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.852621:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346220160 00000100:00100000:0.0:1787826319.852628:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346220160 00000100:00100000:0.0:1787826319.852762:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23693 00000100:00100000:0.0:1787826319.852771:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346220160:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826319.852865:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346220160:12345-0@lo:1000:mdt_io00_001.0 Request processed in 95us (1044us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.852872:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23693 00000100:00100000:1.0:1787826319.852938:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346220160:0@lo:1000:mdt_io00_001.0 00000004:00020000:1.0:1787826319.852979:0:6541:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:1.0:1787826319.853049:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.853059:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.853068:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826319.853083:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346220288:0@lo:101:mdt_io00_001.0 00000004:00020000:2.0:1787826319.853255:0:6528:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4484 00010000:00010000:2.0:1787826319.853323:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853337:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853348:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826319.853359:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbb:0x0]: returning size 6 00010000:00010000:2.0:1787826319.853459:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853467:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853473:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853479:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853485:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826319.853489:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853496:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853509:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794385aa00) 00010000:00010000:2.0:1787826319.853512:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826319.853516:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853522:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4acc3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853536:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853542:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853548:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853552:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826319.853555:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853560:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853566:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794385a500) 00010000:00010000:2.0:1787826319.853569:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826319.853571:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 2/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853576:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4acca lrc: 0/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853641:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4acbc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853666:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.853673:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:2.0:1787826319.853675:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826319.853677:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826319.853730:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225115:x1874663332600192:12345-192.168.203.56@tcp:101:cat.0 Request processed in 516259us (516397us total) trans 8589957395 rc 0/0 00000100:00100000:2.0:1787826319.853737:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68444 00000100:00100000:1.0:1787826319.854137:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:2.0:1787826319.862798:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973848 is committed 00000100:00100000:3.0:1787826319.863326:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346220288 00000100:00100000:3.0:1787826319.863336:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346220288 00000100:00100000:3.0:1787826319.863411:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23694 00000100:00100000:3.0:1787826319.863423:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346220288:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826319.863442:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826319.863465:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ad41 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.863491:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 2/0,1 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.863502:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.863511:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 5/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ad41 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.863545:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.863558:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 12 pid: 35080 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.863574:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b618380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346220416:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826319.863677:0:125377:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:00080000:3.0:1787826319.863716:0:125377:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00000100:00100000:3.0:1787826319.863744:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784548a240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.863809:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.863919:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332602240 00000100:00100000:3.0:1787826319.863924:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332602240 00000100:00100000:3.0:1787826319.863948:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30830 00000100:00100000:3.0:1787826319.863954:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0+-99:2392:x1874663332602240:12345-192.168.203.56@tcp:8:cat.0 00010000:00080000:3.0:1787826319.863981:0:20167:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from 398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp t0 exp 0000000000000000 cur 8132 last 0 00000100:00100000:0.0:1787826319.864173:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aaf00 going to sleep for 11 seconds 00000020:00000080:3.0:1787826319.864511:0:20167:0:(genops.c:1385:class_connect()) connect: client 398e0287-d3c7-44dd-b1d9-6b85f698e615, cookie 0xe21309aa81f4ad5d 00002000:00100000:3.0:1787826319.864524:0:20167:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli 398e0287-d3c7-44dd-b1d9-6b85f698e615/ffff9f784407e000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113950 ocd_grant: 8437760 ocd_index: 0 ocd_group 0 00002000:00100000:3.0:1787826319.864544:0:20167:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:3.0:1787826319.864615:0:20167:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:3.0:1787826319.864648:0:20167:0:(import.c:65:import_set_state_nolock()) ffff9f784407f800 : changing import state from RECOVER to FULL 00000100:00100000:3.0:1787826319.864707:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2392:x1874663332602240:12345-192.168.203.56@tcp:8:cat.0 Request processed in 750us (898us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.864715:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30830 00010000:00010000:2.0:1787826319.869225:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4aaee expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826319.869247:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4aaee expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.869298:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aaee expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.869305:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.869339:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4aae7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aaee expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826319.869362:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985404 is committed 00000020:00080000:2.0:1787826319.869393:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784c4f4a80 refcount 2 committed 0 result 0 batchid 3417 00000020:00080000:2.0:1787826319.869397:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784c4f4d80 00000020:00080000:2.0:1787826319.869401:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1520 00000020:00080000:2.0:1787826319.869403:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1800 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7943eb1000 00000020:00080000:2.0:1787826319.869406:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1202 00000001:00080000:2.0:1787826319.869415:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985401 is committed 00000020:00080000:2.0:1787826319.869419:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7844e81a80 refcount 2 committed 0 result 0 batchid 3416 00000020:00080000:2.0:1787826319.869421:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e81180 00000020:00080000:2.0:1787826319.869424:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1519 00000020:00080000:2.0:1787826319.869425:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79522a2b00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78491ac200 00000020:00080000:2.0:1787826319.869428:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1201 00000004:00080000:2.0:1787826319.869436:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 403 committed 00000020:00080000:2.0:1787826319.869444:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7844e810c0 refcount 2 committed 0 result 0 batchid 3415 00000020:00080000:2.0:1787826319.869447:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e81900 00000020:00080000:2.0:1787826319.869449:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1518 00000020:00080000:2.0:1787826319.869451:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7946078600 00000020:00080000:2.0:1787826319.869453:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1200 00000020:00080000:2.0:1787826319.869461:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784d9b1e40 refcount 2 committed 0 result 0 batchid 3414 00000020:00080000:2.0:1787826319.869463:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d9b13c0 00000020:00080000:2.0:1787826319.869466:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1517 00000020:00080000:2.0:1787826319.869468:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784cf5bc00 00000020:00080000:2.0:1787826319.869470:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1199 00000001:00080000:2.0:1787826319.869475:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985395 is committed 00000020:00080000:2.0:1787826319.869490:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3413 batchid updated 00000100:00100000:0.0:1787826319.869719:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346220544:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826319.869803:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826319.870196:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870206:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870213:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870219:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826319.870223:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870229:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870244:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:3.0:1787826319.870247:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826319.870251:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870256:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4aba4 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870269:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870275:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870281:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870286:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826319.870289:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870295:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870302:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:3.0:1787826319.870306:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826319.870311:0:3638:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ad41 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870321:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870326:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4abab lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870339:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870344:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870349:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870353:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826319.870356:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870361:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870629:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x4ec94fd6 (ffff9f7843326500) 00010000:00010000:3.0:1787826319.870632:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826319.870635:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.870642:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ac4c lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787826319.870675:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3417 committed batchid 3413 00000004:00080000:3.0:1787826319.870786:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784548af00 with version 6834 00000020:00080000:3.0:1787826319.870799:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3412 cancel update log [0x200000401:0x3:0x0].1515: rc = 0 00000020:00080000:3.0:1787826319.870844:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3412 cancel update log [0x240000400:0x3:0x0].1197: rc = 0 00000100:00100000:2.0:1787826319.870972:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346220544 00000100:00100000:2.0:1787826319.870981:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346220544 00000100:00100000:2.0:1787826319.871034:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11101 00000100:00100000:2.0:1787826319.871042:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346220544:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826319.871056:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826319.871073:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4aaee lrc: 0/0,0 mode: --/PW res: [0x200000401:0x192:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4aae7 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.871086:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x192:0x0].0x5a734ce9 (ffff9f7843326600) 00010000:00010000:2.0:1787826319.871093:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:3.0:1787826319.872060:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846d58d80 with version 6835 00000020:00080000:3.0:1787826319.872068:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3413 cancel update log [0x200000401:0x3:0x0].1516: rc = 0 00000100:00100000:2.0:1787826319.872269:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346220544:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 1224us (2463us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.872278:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11101 00000100:00100000:0.0:1787826319.872445:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346220544:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826319.872547:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826319.872572:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332602496 00000100:00100000:0.0:1787826319.872578:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332602496 00010000:00010000:0.0:1787826319.872625:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 7 pid: 35080 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.872653:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21952 00000100:00100000:0.0:1787826319.872663:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971597850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332602496:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826319.872675:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826319.872681:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 7 pid: 35080 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.872693:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 7 pid: 35080 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.872713:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:0.0:1787826319.872718:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ad41 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.872728:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4ad41 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.872736:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4ad41 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.872772:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ad41 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.872786:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6512:1874663346220800:0@lo:105: 00000100:00100000:0.0:1787826319.872822:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826319.872854:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fa80 going to sleep for 11 seconds 00010000:00010000:2.0:1787826319.874041:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4aca7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801513 expref: 7 pid: 35080 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.874064:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b618380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346220416:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826319.874094:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ad41 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.874130:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:2.0:1787826319.874133:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826319.874137:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:2.0:1787826319.874206:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6541:x1874663346220288:12345-0@lo:101:mdt_io00_001.0 Request processed in 10786us (20068us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.874213:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23694 00000100:00100000:1.0:1787826319.874419:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346220672:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1787826319.874463:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1787826319.874835:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6834 rpc_version 6834 00000020:00100000:1.0:1787826319.874887:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826319.874895:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784836d880 x1874663346221056/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826319.874914:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346221056:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826319.874949:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.874970:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.875207:0:6541:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985395 gen 2 00000004:00080000:0.0:1787826319.875214:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985394 committed transno 4294985395 00000020:00080000:0.0:1787826319.875221:0:6541:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f784f7d73c0 refcount 2 committed 0 result 0 batchid 4294979082 00000020:00080000:0.0:1787826319.875225:0:6541:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cf3ae00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f7d7e40 00000020:00080000:0.0:1787826319.875228:0:6541:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].938 00000020:00080000:0.0:1787826319.875231:0:6541:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f794b603e00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a190c00 00000020:00080000:0.0:1787826319.875234:0:6541:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].935 00000020:00080000:1.0:1787826319.875278:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979082 committed batchid 4294979081 00000020:00080000:1.0:1787826319.875347:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979081 00000004:00080000:0.0:1787826319.875766:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985395 committed transno 4294985395 00000100:00100000:0.0:1787826319.875785:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346220288:0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826319.875809:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.875820:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.875919:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346220672 00000100:00100000:2.0:1787826319.875927:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346220672 00000100:00100000:2.0:1787826319.875993:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30831 00000100:00100000:2.0:1787826319.876003:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346220672:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:2.0:1787826319.876031:0:8443:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1226 count 1 00010000:00010000:2.0:1787826319.876055:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.876073:0:8443:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.876081:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4ca:0x0].0x0 since=11s v=(0 0) 00010000:00010000:2.0:1787826319.876085:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826319.876089:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.876146:0:8443:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.876161:0:8443:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 6 pid: 35096 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.876173:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8443:1874663346221184:192.168.203.56@tcp:104: 00000100:00080000:0.0:1787826319.876263:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000020:00080000:3.0:1787826319.876342:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3413 cancel update log [0x240000400:0x3:0x0].1198: rc = 0 00000020:00080000:3.0:1787826319.876353:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3413 00000100:00100000:3.0:1787826319.876393:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346220800 00000100:00100000:3.0:1787826319.876397:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346220800 00000100:00100000:3.0:1787826319.876451:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7868 00000100:00100000:3.0:1787826319.876459:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6512:x1874663346220800:12345-0@lo:105: 00010000:00010000:3.0:1787826319.876469:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.876478:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.876486:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.876494:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.876502:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.876510:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.876526:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece800) 00000100:00100000:3.0:1787826319.876597:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6512:x1874663346220800:12345-0@lo:105: Request processed in 129us (3764us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.876605:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7868 00010000:00010000:0.0:1787826319.876643:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.876651:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826319.876681:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x195:0x0] 00010000:00010000:0.0:1787826319.876754:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.876762:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.876771:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.876788:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346221312:0@lo:101:mdt_io00_001.0 00000100:00100000:0.0:1787826319.876821:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826319.876842:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.876859:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346221056 00000100:00100000:0.0:1787826319.876863:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346221056 00000100:00100000:0.0:1787826319.876891:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346221312 00000100:00100000:0.0:1787826319.876894:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346221312 00000100:00100000:0.0:1787826319.876911:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23695 00000100:00100000:0.0:1787826319.876917:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346221056:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826319.877049:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346221056:12345-0@lo:1000:osp_up0-1.0 Request processed in 130us (2098us total) trans 8589957396 rc 0/0 00000100:00100000:0.0:1787826319.877056:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23695 00000100:00100000:0.0:1787826319.877071:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23696 00000100:00100000:0.0:1787826319.877077:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6541:x1874663346221312:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826319.877086:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826319.877102:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ad72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ad6b expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.877142:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ad72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ad6b expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.877153:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ad72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ad6b expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.877161:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x4ec94fd6 (ffff9f784b4a5200) 00010000:00010000:0.0:1787826319.877164:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826319.877167:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826319.877196:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346221312:12345-0@lo:101:mdt_io00_001.0 Request processed in 119us (374us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.877201:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23696 00000100:00100000:0.0:1787826319.877252:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346221312:0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826319.877264:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ad72 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.877277:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ad72 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826319.877291:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x195:0x0] 00010000:00010000:0.0:1787826319.877307:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.877318:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.877325:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.877344:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.877354:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 10 pid: 35092 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.877363:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346221440:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826319.877471:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6512:1874663346220800:0@lo:105: 00010000:00010000:3.0:1787826319.877484:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826319.877802:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.877988:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.878022:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971597850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332602496:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 5359us (5476us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.878033:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21952 00000100:00100000:1.0:1787826319.880332:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.880618:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332602368 00000100:00100000:3.0:1787826319.880627:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332602368 00000100:00100000:3.0:1787826319.880695:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30832 00000100:00100000:3.0:1787826319.880706:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332602368:12345-192.168.203.56@tcp:101:cat.0 00000100:00080000:0.0:1787826319.880722:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787826319.880725:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826319.880746:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4ad80 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080154b expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.880806:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346221568:192.168.203.56@tcp:106: 00000100:00100000:3.0:1787826319.880988:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd16c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826319.881182:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346221056:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826319.881197:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6835 rpc_version 6835 00000020:00100000:1.0:1787826319.881214:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826319.881221:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784836ea00 x1874663346221824/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826319.881238:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346221824:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826319.881279:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.881305:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.881544:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346221824 00000100:00100000:2.0:1787826319.881555:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346221824 00000100:00100000:1.0:1787826319.881657:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23697 00000100:00100000:1.0:1787826319.881670:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346221824:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826319.881969:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346221824:12345-0@lo:1000:osp_up0-1.0 Request processed in 302us (688us total) trans 8589957397 rc 0/0 00000100:00100000:3.0:1787826319.882069:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346221824:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826319.886190:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826319.886222:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826319.886289:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346221440:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826319.886317:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.886352:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332602752 00000100:00100000:0.0:1787826319.886357:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332602752 00010000:00010000:0.0:1787826319.886393:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 11 pid: 35092 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.886435:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332602624 00000100:00100000:0.0:1787826319.886439:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332602624 00010000:00010000:0.0:1787826319.886454:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 7 pid: 35096 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.886482:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21953 00000100:00100000:0.0:1787826319.886492:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2395:x1874663332602752:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826319.886507:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826319.886514:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 11 pid: 35092 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886526:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 11 pid: 35092 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886551:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4acb5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080151a expref: 11 pid: 35092 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886582:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794385aa00) 00010000:00010000:0.0:1787826319.886599:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886610:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886620:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886642:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826319.886704:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2395:x1874663332602752:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 212us (525us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.886712:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21953 00000100:00100000:0.0:1787826319.886732:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21954 00000100:00100000:0.0:1787826319.886739:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332602624:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826319.886750:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826319.886756:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 7 pid: 35096 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886768:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 7 pid: 35096 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886789:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826319.886793:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826319.886799:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886809:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886821:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826319.886839:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332602624:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 100us (616us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.886846:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21954 00010000:00010000:0.0:1787826319.886944:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886952:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886972:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886982:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.886990:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.887055:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346221952:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826319.887132:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826319.887159:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.887297:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346221952 00000100:00100000:3.0:1787826319.887302:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346221952 00010000:00010000:2.0:1787826319.889549:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005896499e/0xe21309aa81f4a90b lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801410 expref: 6 pid: 35096 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.889582:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8443:1874663346221184:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826319.889620:0:8443:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.889632:0:8443:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.889639:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.889651:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.889659:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.889686:0:8443:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826319.889972:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346220672:12345-0@lo:6:osp-syn-1-1.0 Request processed in 13968us (15507us total) trans 4294976157 rc 0/0 00000100:00100000:2.0:1787826319.889981:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30831 00010000:00010000:2.0:1787826319.890012:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826319.890019:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.890033:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efc00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826319.890038:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.890047:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.890067:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.890074:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010d0e8fb/0xe21309aa81f4ad64 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.890093:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826319.890094:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826319.890096:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1787826319.890453:0:17826:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x2c0000401:0x520:0x0] updating lvb size: 0 -> 6 00002000:00010000:0.0:1787826319.890458:0:17826:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x2c0000401:0x520:0x0] updating lvb mtime: 0 -> 1787826314 00002000:00010000:0.0:1787826319.890461:0:17826:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x2c0000401:0x520:0x0] updating lvb atime: 0 -> 1787826314 00002000:00010000:0.0:1787826319.890464:0:17826:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x2c0000401:0x520:0x0] updating lvb ctime: 0 -> 1787826314 00002000:00010000:0.0:1787826319.890466:0:17826:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x2c0000401:0x520:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:0.0:1787826319.890482:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346221568:192.168.203.56@tcp:106: 00010000:00010000:0.0:1787826319.890511:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4ad80 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080154b expref: 7 pid: 17826 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826319.891371:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23697 00000004:00080000:1.0:1787826319.891432:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784d009180/2, rc 0, transno 4294976157 00000100:00100000:1.0:1787826319.891438:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346220672:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1787826319.891571:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23698 00000100:00100000:3.0:1787826319.891599:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346221952:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.891713:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346221952:12345-0@lo:1000:mdt_io00_001.0 Request processed in 130us (4606us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.891721:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23698 00000100:00100000:1.0:1787826319.891802:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346221952:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.891884:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222080:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.891927:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.892039:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.892256:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346222080 00000100:00100000:2.0:1787826319.892264:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346222080 00000100:00100000:2.0:1787826319.892302:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23699 00000100:00100000:2.0:1787826319.892311:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222080:12345-0@lo:1000:mdt_io00_001.0 00010000:00010000:0.0:1787826319.892321:0:17826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4ad80 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080154b expref: 7 pid: 17826 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.892341:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:0.0:1787826319.892402:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332602368:12345-192.168.203.56@tcp:101:cat.0 Request processed in 11698us (12078us total) trans 0 rc 301/301 00000100:00100000:0.0:1787826319.892409:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30832 00000100:00100000:2.0:1787826319.892621:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222080:12345-0@lo:1000:mdt_io00_001.0 Request processed in 310us (693us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.892629:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23699 00000100:00100000:1.0:1787826319.892679:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222080:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.892730:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222208:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.892763:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.892788:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.892822:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346222208 00000100:00100000:0.0:1787826319.892828:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346222208 00000100:00100000:0.0:1787826319.892869:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23700 00000100:00100000:0.0:1787826319.892877:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222208:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826319.893016:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222208:12345-0@lo:1000:mdt_io00_001.0 Request processed in 139us (251us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826319.893023:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23700 00000100:00100000:1.0:1787826319.894052:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222208:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.894167:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222336:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.894209:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.894236:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.894541:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346222336 00000100:00100000:0.0:1787826319.894548:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346222336 00000100:00100000:0.0:1787826319.894598:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23701 00000100:00100000:0.0:1787826319.894607:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222336:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826319.894700:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222336:12345-0@lo:1000:mdt_io00_001.0 Request processed in 91us (487us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.894709:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23701 00000100:00100000:1.0:1787826319.895261:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222336:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.895352:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222464:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.895391:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.895428:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.895474:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346222464 00000100:00100000:0.0:1787826319.895479:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346222464 00000100:00100000:0.0:1787826319.895515:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23702 00000100:00100000:0.0:1787826319.895524:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222464:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826319.895609:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222464:12345-0@lo:1000:mdt_io00_001.0 Request processed in 85us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.895617:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23702 00000100:00100000:1.0:1787826319.895797:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222464:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826319.895820:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4e:0x0] 00010000:00010000:1.0:1787826319.895913:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.895932:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.895942:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.895968:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.895984:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 9 pid: 6528 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826319.895997:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346222592:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826319.896088:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.897085:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826319.897233:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332602880 00000100:00100000:0.0:1787826319.897241:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332602880 00000100:00100000:0.0:1787826319.897290:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30833 00000100:00100000:0.0:1787826319.897300:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225115:x1874663332602880:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826319.897316:0:35097:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826319.897335:0:35097:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801552 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.897362:0:35097:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.897370:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x520:0x0].0x0 since=5s v=(0 0) 00010000:00010000:0.0:1787826319.897375:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826319.897379:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 7 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.897405:0:35097:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.897414:0:35097:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 5 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.897446:0:35097:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:35097:1874663346222720:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826319.897518:0:35097:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.900363:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.900695:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332603008 00000100:00100000:3.0:1787826319.900927:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332603008 00010000:00010000:3.0:1787826319.900976:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 10 pid: 6528 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.901066:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21955 00000100:00100000:3.0:1787826319.901078:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332603008:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826319.901094:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826319.901102:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 10 pid: 6528 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.901159:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 10 pid: 6528 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.901188:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:3.0:1787826319.901195:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.901205:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.901216:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.901236:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826319.901459:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332603008:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 381us (1104us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.901469:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21955 00010000:00010000:1.0:1787826319.901568:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ad48 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801521 expref: 9 pid: 6528 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826319.901598:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346222592:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826319.901622:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.901632:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.901638:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.901694:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.901702:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.901713:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826319.901733:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222848:0@lo:101:mdt_io00_001.0 00000100:00100000:1.0:1787826319.901774:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.901810:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.901834:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346222848 00000100:00100000:2.0:1787826319.901841:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346222848 00000100:00100000:2.0:1787826319.901894:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23703 00000100:00100000:2.0:1787826319.901902:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346222848:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:2.0:1787826319.901919:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826319.901934:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4ada3 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ad9c expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.901956:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4ada3 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ad9c expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.901969:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4ada3 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ad9c expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.901976:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7967ac8a00) 00010000:00010000:2.0:1787826319.901979:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787826319.901992:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826319.902026:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346222848:12345-0@lo:101:mdt_io00_001.0 Request processed in 125us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.902033:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23703 00000100:00100000:1.0:1787826319.903022:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222848:0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826319.903051:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ada3 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.903063:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ada3 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826319.903085:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4f:0x0] 00000100:00100000:1.0:1787826319.903157:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222976:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.903205:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.903232:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826319.903255:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346222976 00000100:00100000:1.0:1787826319.903260:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346222976 00000100:00100000:1.0:1787826319.903304:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23704 00000100:00100000:1.0:1787826319.903314:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346222976:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.903780:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346222976:12345-0@lo:1000:mdt_io00_001.0 Request processed in 465us (573us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.903789:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23704 00000100:00100000:1.0:1787826319.904373:0:35097:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:35097:1874663346222720:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826319.904405:0:35097:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 7 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.904438:0:35097:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:1.0:1787826319.904494:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225115:x1874663332602880:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7196us (7415us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.904502:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30833 00000100:00100000:2.0:1787826319.905321:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346222976:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826319.905364:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x195:0x0] 00000100:00100000:2.0:1787826319.905438:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223104:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826319.905492:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826319.905525:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d79c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.905602:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826319.907193:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346223104 00000100:00100000:1.0:1787826319.907206:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346223104 00000100:00100000:1.0:1787826319.907265:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23705 00000100:00100000:1.0:1787826319.907275:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223104:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.907432:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223104:12345-0@lo:1000:mdt_io00_001.0 Request processed in 147us (1928us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.907440:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23705 00000100:00100000:2.0:1787826319.908585:0:20169:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332603136 00000100:00100000:2.0:1787826319.908608:0:20169:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332603136 00002000:00100000:2.0:1787826319.908663:0:20169:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x520:0x0] (0->5) req@ffff9f784d00a680 x1874663332603136/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:675/0 lens 496/0 e 0 to 0 dl 1787826330 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826319.908680:0:20169:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d00a680 with x1874663332603136 ext(0->5) 00010000:00010000:2.0:1787826319.908689:0:20169:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 6 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.908705:0:20169:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 6 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826319.908712:0:20169:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9f784d00a680 00000100:00100000:2.0:1787826319.908737:0:20169:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2490 00000100:00100000:2.0:1787826319.908746:0:20169:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332603136:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826319.910335:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223104:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826319.910384:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4e:0x0] 00010000:00010000:3.0:1787826319.910442:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.910456:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.910468:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.910560:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223232:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.910634:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826319.910673:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd19c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.910741:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346223232 00000100:00100000:0.0:1787826319.910747:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346223232 00000100:00100000:0.0:1787826319.910798:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23706 00000100:00100000:0.0:1787826319.910808:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223232:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826319.910889:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223232:12345-0@lo:1000:mdt_io00_001.0 Request processed in 82us (255us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.910897:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23706 00000100:00100000:3.0:1787826319.912396:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223232:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826319.912438:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4e:0x0] 00000100:00100000:3.0:1787826319.912487:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223360:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.912540:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826319.912574:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd19c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.916265:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346223360 00000100:00100000:2.0:1787826319.916281:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346223360 00000100:00100000:2.0:1787826319.916356:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23707 00000100:00100000:2.0:1787826319.916366:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223360:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826319.916520:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223360:12345-0@lo:1000:mdt_io00_001.0 Request processed in 156us (3979us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.916528:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23707 00000100:00100000:3.0:1787826319.922787:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223360:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.922890:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223488:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.922956:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826319.923007:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd19c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826319.926346:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346223488 00000100:00100000:1.0:1787826319.926362:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346223488 00000100:00100000:1.0:1787826319.926456:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23708 00000100:00100000:1.0:1787826319.926468:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223488:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.926639:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223488:12345-0@lo:1000:mdt_io00_001.0 Request processed in 172us (3681us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.926647:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23708 00000100:00100000:2.0:1787826319.926968:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b619c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223488:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826319.927027:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x195:0x0] 00000100:00100000:2.0:1787826319.927082:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223616:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826319.927172:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826319.927198:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826319.927256:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346223616 00000100:00100000:1.0:1787826319.927261:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346223616 00000100:00100000:1.0:1787826319.927299:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23709 00000100:00100000:1.0:1787826319.927308:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223616:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.927379:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223616:12345-0@lo:1000:mdt_io00_001.0 Request processed in 72us (207us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.927385:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23709 00000100:00100000:3.0:1787826319.927456:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223616:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826319.927482:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4e:0x0] 00000100:00100000:3.0:1787826319.927519:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223744:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.927548:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826319.927575:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd1480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826319.927689:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346223744 00000100:00100000:1.0:1787826319.927694:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346223744 00000100:00100000:1.0:1787826319.927730:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23710 00000100:00100000:1.0:1787826319.927737:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223744:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.927789:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223744:12345-0@lo:1000:mdt_io00_001.0 Request processed in 53us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.927794:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23710 00000100:00100000:3.0:1787826319.927860:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223744:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.927890:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850359880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223872:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.927911:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826319.927933:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd1480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826319.927984:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346223872 00000100:00100000:1.0:1787826319.927988:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346223872 00000100:00100000:1.0:1787826319.928017:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23711 00000100:00100000:1.0:1787826319.928023:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223872:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826319.928067:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346223872:12345-0@lo:1000:mdt_io00_001.0 Request processed in 44us (154us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.928073:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23711 00000100:00100000:0.0:1787826319.928280:0:20169:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332603136:12345-192.168.203.56@tcp:4:bash.0 Request processed in 19532us (22686us total) trans 4294976158 rc 0/0 00000100:00100000:0.0:1787826319.928289:0:20169:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2490 00002000:00100000:0.0:1787826319.928300:0:20169:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x520:0x0] (0->5) req@ffff9f784d00a680 x1874663332603136/t4294976158(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:675/0 lens 496/448 e 0 to 0 dl 1787826330 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826319.928311:0:20169:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d00a680 with x1874663332603136 ext(0->5) 00010000:00010000:0.0:1787826319.928321:0:20169:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 8 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.928340:0:20169:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 8 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826319.928349:0:20169:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9f784d00a680 00000100:00100000:3.0:1787826319.928693:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850359880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346223872:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.928771:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850359880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346224000:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.928800:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826319.928827:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd1480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.928843:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346224000 00000100:00100000:3.0:1787826319.928848:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346224000 00000100:00100000:3.0:1787826319.928881:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23712 00000100:00100000:3.0:1787826319.928889:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346224000:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826319.928952:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346224000:12345-0@lo:1000:mdt_io00_001.0 Request processed in 65us (151us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826319.928959:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23712 00000100:00100000:3.0:1787826319.928984:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850359880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346224000:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826319.929378:0:6541:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1dbb:0x0] is inserted into orphan list, open count = 1 00000040:00080000:3.0:1787826319.929748:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].936, 1240 off880728 00000040:00080000:3.0:1787826319.929800:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].939, 1320 off979952 00000004:00080000:3.0:1787826319.929806:0:6541:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7845dd1240 with version 1826 00000004:00080000:1.0:1787826319.929995:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1826 rpc_version 1826 00000020:00100000:2.0:1787826319.930294:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826319.930304:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c95500 x1874663346224256/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.930322:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95500 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346224256:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826319.930365:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826319.930573:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d79c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.930621:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346224256 00000100:00100000:3.0:1787826319.930627:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346224256 00000100:00100000:3.0:1787826319.930699:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23713 00000100:00100000:3.0:1787826319.930709:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346224256:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826319.931085:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346224256:12345-0@lo:1000:osp_up1-0.0 Request processed in 377us (717us total) trans 4294985405 rc 0/0 00000100:00100000:3.0:1787826319.931093:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23713 00000100:00100000:1.0:1787826319.931199:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95500 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346224256:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787826319.931337:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931348:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931357:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931363:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826319.931368:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931375:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931390:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:0.0:1787826319.931393:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826319.931398:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931404:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4ad8e lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826319.931436:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e4f:0x0] 00010000:00010000:0.0:1787826319.931442:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ada3 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931451:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ada3 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931464:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931470:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931481:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931486:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826319.931488:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931494:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931501:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794385aa00) 00010000:00010000:0.0:1787826319.931503:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826319.931506:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931511:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ad79 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826319.931523:0:6541:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006434dafd reply state 00000000fd7b7730 transno 8589957398 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931535:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931542:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794385a500) 00010000:00010000:0.0:1787826319.931544:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826319.931546:0:6541:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:0.0:1787826319.931549:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x195:0x0] 00010000:00010000:0.0:1787826319.931553:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931566:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931573:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931606:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826319.931610:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x195:0x0] 00010000:00010000:0.0:1787826319.931615:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ad72 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931622:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ad72 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931630:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931636:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931642:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931647:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826319.931650:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931656:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931665:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7843326500) 00010000:00010000:0.0:1787826319.931667:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826319.931670:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.931676:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4adaa lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.931746:0:6541:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b619880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225117:x1874663332602112:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 82902us (83984us total) trans 8589957398 rc 0/0 00000100:00100000:0.0:1787826319.931753:0:6541:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1230 00010000:00010000:1.0:1787826319.931904:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826319.931910:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.931917:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826319.931922:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826319.931930:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.931955:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.931961:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826319.931987:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.931994:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4ad41 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ad56 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.932008:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826319.932009:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826319.932012:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826319.932019:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826319.932031:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346224384:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826319.932084:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826319.932148:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346224384 00000100:00100000:1.0:1787826319.932153:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346224384 00000100:00100000:1.0:1787826319.932191:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11102 00000100:00100000:1.0:1787826319.932198:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346224384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826319.932210:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826319.932225:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ad56 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ad41 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.932236:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f7843326c00) 00010000:00010000:1.0:1787826319.932242:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826319.932266:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346224384:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (182us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.932272:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11102 00000100:00100000:1.0:1787826319.932299:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346224384:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826319.938371:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826319.938542:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332603264 00000100:00100000:2.0:1787826319.938553:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332603264 00010000:00010000:2.0:1787826319.938612:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 8 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.938694:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21956 00000100:00100000:2.0:1787826319.938705:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d24000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332603264:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826319.938722:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1787826319.938744:0:6511:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000401:0x520:0x0] updating lvb blocks from disk: 1 -> 8 00010000:00010000:2.0:1787826319.938751:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 8 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.938769:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 8 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.938796:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4ac37 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014e9 expref: 8 pid: 8445 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.938813:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826319.938817:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826319.938826:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.938840:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.938880:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.938896:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6511:1874663346224512:192.168.203.56@tcp:105: 00000100:00100000:2.0:1787826319.938997:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d79c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.950330:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826319.950646:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6511:1874663346224512:192.168.203.56@tcp:105: 00010000:00010000:2.0:1787826319.950673:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826319.950736:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d24000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332603264:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 12032us (12375us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826319.950747:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21956 00000100:00100000:1.0:1787826319.950847:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332603392 00000100:00100000:1.0:1787826319.950854:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332603392 00000100:00100000:1.0:1787826319.950917:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68450 00000100:00100000:1.0:1787826319.950927:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225115:x1874663332603392:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826319.950949:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826319.950972:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4adb1 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801559 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.951049:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4adb1 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801559 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.951063:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4adb1 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801559 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.951082:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f794385ae00) 00010000:00010000:1.0:1787826319.951085:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787826319.951088:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826319.951157:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225115:x1874663332603392:12345-192.168.203.56@tcp:101:cat.0 Request processed in 232us (838us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826319.951164:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68450 00000100:00100000:1.0:1787826319.965354:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.968015:0:38010:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332603520 00000100:00100000:3.0:1787826319.968033:0:38010:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332603520 00002000:00100000:3.0:1787826319.968102:0:38010:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_013: refresh rw locks for [0x2c0000401:0x520:0x0] (0->4095) req@ffff9f79785e5f80 x1874663332603520/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:675/0 lens 488/0 e 0 to 0 dl 1787826330 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826319.968129:0:38010:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f79785e5f80 with x1874663332603520 ext(0->4095) 00002000:00010000:3.0:1787826319.968135:0:38010:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f79785e5f80 00000100:00100000:3.0:1787826319.968969:0:38010:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27648 00000100:00100000:3.0:1787826319.968986:0:38010:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332603520:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:3.0:1787826319.969203:0:38010:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332603520:12345-192.168.203.56@tcp:3:cat.0 Request processed in 222us (3859us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787826319.969211:0:38010:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27648 00002000:00100000:3.0:1787826319.969219:0:38010:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_013: refresh rw locks for [0x2c0000401:0x520:0x0] (0->4095) req@ffff9f79785e5f80 x1874663332603520/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:675/0 lens 488/4536 e 0 to 0 dl 1787826330 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826319.969229:0:38010:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f79785e5f80 with x1874663332603520 ext(0->4095) 00002000:00010000:3.0:1787826319.969235:0:38010:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f79785e5f80 00000100:00100000:0.0:1787826319.982406:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826319.982948:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332603776 00000100:00100000:3.0:1787826319.982967:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332603776 00000100:00100000:2.0:1787826319.983660:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9347 00000100:00100000:2.0:1787826319.983683:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225115:x1874663332603776:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:2.0:1787826319.983925:0:6530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1dbb:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1787826319.984003:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].213, 64 off21824 00000004:00080000:2.0:1787826319.984016:0:6530:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8132195909715ns, next at 8133195905884ns, rc = 0 00010000:00010000:2.0:1787826319.984074:0:6530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.984089:0:6530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.984147:0:6530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.984155:0:6530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.984161:0:6530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.984167:0:6530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.984184:0:6530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826319.984293:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225115:x1874663332603776:12345-192.168.203.56@tcp:35:cat.0 Request processed in 618us (1896us total) trans 8589957399 rc 0/0 00000100:00100000:2.0:1787826319.984302:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9347 00010000:00010000:1.0:1787826319.987703:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826319.987718:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.987730:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826319.987735:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.987745:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.987765:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f794385ae00) 00010000:00010000:1.0:1787826319.987768:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826319.987773:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.987778:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4adb8 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dbb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.987801:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826319.987803:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826319.987805:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826320.364422:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346224768:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364510:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364553:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346225280:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364576:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364605:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346225792:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364629:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364656:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1787826320.364662:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346224640:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364695:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364724:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346225152:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364749:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364768:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346225664:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364789:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364816:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346224896:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364840:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364857:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346225408:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364878:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364897:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1787826320.364902:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346225024:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364927:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.364944:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346225536:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.364966:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826320.365005:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346224768 00000100:00100000:0.0:1787826320.365014:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346224768 00000100:00100000:0.0:1787826320.365069:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346225280 00000100:00100000:0.0:1787826320.365073:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346225280 00000100:00100000:0.0:1787826320.365101:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346225792 00000100:00100000:0.0:1787826320.369102:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346225792 00000100:00100000:0.0:1787826320.369290:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346224896 00000100:00100000:0.0:1787826320.369296:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346224896 00000100:00100000:0.0:1787826320.369325:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346225408 00000100:00100000:0.0:1787826320.369329:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346225408 00000100:00100000:0.0:1787826320.369351:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346225024 00000100:00100000:0.0:1787826320.369354:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346225024 00000100:00100000:0.0:1787826320.369384:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14302 00000100:00100000:0.0:1787826320.369397:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346224768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.369522:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346224768:12345-0@lo:400:kworker.0 Request processed in 128us (5012us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826320.369530:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14302 00000100:00100000:0.0:1787826320.369552:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14303 00000100:00100000:0.0:1787826320.369560:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346225280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.369607:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346225280:12345-0@lo:400:kworker.0 Request processed in 48us (5029us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826320.369612:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14303 00000100:00100000:0.0:1787826320.369627:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14304 00000100:00100000:0.0:1787826320.369633:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346225792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.369662:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346225792:12345-0@lo:400:kworker.0 Request processed in 30us (5032us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826320.369667:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14304 00000100:00100000:0.0:1787826320.369678:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14305 00000100:00100000:0.0:1787826320.369685:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346224896:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.369715:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346224896:12345-0@lo:400:kworker.0 Request processed in 31us (4874us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826320.369721:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14305 00000100:00100000:0.0:1787826320.369735:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14306 00000100:00100000:0.0:1787826320.369741:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346225408:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.369769:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346225408:12345-0@lo:400:kworker.0 Request processed in 29us (4890us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826320.369774:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14306 00000100:00100000:0.0:1787826320.369784:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14307 00000100:00100000:0.0:1787826320.369790:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346225024:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.369814:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346225024:12345-0@lo:400:kworker.0 Request processed in 25us (4887us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826320.369820:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14307 00000100:00100000:3.0:1787826320.373488:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346225152 00000100:00100000:3.0:1787826320.373510:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346225152 00000100:00100000:3.0:1787826320.373578:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346225664 00000100:00100000:3.0:1787826320.373582:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346225664 00000100:00100000:3.0:1787826320.373617:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346225536 00000100:00100000:3.0:1787826320.373620:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346225536 00000100:00100000:3.0:1787826320.373650:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8822 00000100:00100000:3.0:1787826320.373662:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346225152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826320.373774:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346225152:12345-0@lo:400:kworker.0 Request processed in 115us (9024us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826320.373783:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8822 00000100:00100000:3.0:1787826320.373804:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8823 00000100:00100000:3.0:1787826320.373811:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346225664:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826320.373842:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346225664:12345-0@lo:400:kworker.0 Request processed in 32us (9051us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826320.373849:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8823 00000100:00100000:3.0:1787826320.373875:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8824 00000100:00100000:3.0:1787826320.373882:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346225536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826320.373910:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346225536:12345-0@lo:400:kworker.0 Request processed in 29us (8943us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826320.373916:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8824 00000100:00100000:3.0:1787826320.373957:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346224640 00000100:00100000:3.0:1787826320.373961:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346224640 00000100:00100000:3.0:1787826320.373993:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2225 00000100:00100000:3.0:1787826320.374000:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346224640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826320.374035:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346224640:12345-0@lo:400:kworker.0 Request processed in 36us (9339us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826320.374041:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2225 00000100:00080000:3.0:1787826320.374163:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826320.374168:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826320.374171:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826320.374175:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826320.375183:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346225024:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375198:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973848 gen 2 00000004:00080000:0.0:1787826320.375204:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f796cdb0a80, transno 4294973848 00000100:00100000:0.0:1787826320.375217:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346225536:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375230:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346224896:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375241:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346225408:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375303:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346224640:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375314:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346225152:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375325:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346225664:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375338:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346224768:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375348:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346225280:0@lo:400:kworker.0 00000100:00100000:0.0:1787826320.375360:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346225792:0@lo:400:kworker.0 00010000:00010000:1.0:1787826320.377605:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ad72 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826320.377631:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ad72 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.377695:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ad72 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.377701:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826320.377739:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4ad6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ad72 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.377765:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ada3 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826320.377771:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ada3 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.377787:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ada3 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.377793:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826320.377797:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfe00/0xe21309aa81f4ad9c lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ada3 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826320.377813:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957399 is committed 00000004:00080000:1.0:1787826320.377821:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 218 committed 00000100:00100000:3.0:1787826320.378002:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346226048:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826320.378082:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:1.0:1787826320.378142:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957398 is committed 00000001:00080000:1.0:1787826320.378156:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957397 is committed 00000020:00080000:1.0:1787826320.378168:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979082 batchid updated 00000100:00100000:2.0:1787826320.378315:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346225920:0@lo:103:jbd2/dm-0-8.0 00000020:00080000:2.0:1787826320.378386:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979082 00000004:00080000:2.0:1787826320.378452:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784f7d7e40 with version 1827 00000020:00080000:2.0:1787826320.378463:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979082 cancel update log [0x240000404:0x2:0x0].938: rc = 0 00000020:00080000:2.0:1787826320.378547:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979082 cancel update log [0x200000bd0:0x2:0x0].935: rc = 0 00000020:00080000:2.0:1787826320.378558:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979082 00010000:00010000:2.0:1787826320.378572:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378581:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378609:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378617:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826320.378622:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378631:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378647:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794385a500) 00010000:00010000:2.0:1787826320.378650:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826320.378655:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378661:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4ad87 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378685:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378693:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378699:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378705:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826320.378708:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378719:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378729:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794385a500) 00010000:00010000:2.0:1787826320.378731:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826320.378734:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.378741:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ac22 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826320.378778:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1827 rpc_version 1827 00000020:00100000:2.0:1787826320.378804:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826320.378814:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c95c00 x1874663346226432/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826320.378844:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346226432:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826320.378894:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826320.378932:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d73c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826320.378965:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346226432 00000100:00100000:2.0:1787826320.378974:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346226432 00000100:00100000:1.0:1787826320.380053:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f687800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346226176:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1787826320.380158:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826320.380199:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826320.380242:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346226176 00000100:00100000:1.0:1787826320.380250:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346226176 00000100:00100000:1.0:1787826320.380296:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30834 00000100:00100000:1.0:1787826320.380306:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346226176:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1787826320.380342:0:8443:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1312 count 1 00010000:00010000:1.0:1787826320.380372:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.380399:0:8443:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.380409:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826320.380414:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826320.380420:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.380454:0:8443:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 6 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.380504:0:8443:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 6 pid: 35097 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826320.380517:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8443:1874663346226560:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826320.380611:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23714 00000100:00100000:2.0:1787826320.380621:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346226432:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826320.380683:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aaf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826320.381002:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346226432:12345-0@lo:1000:osp_up1-0.0 Request processed in 382us (2106us total) trans 4294985406 rc 0/0 00000100:00100000:2.0:1787826320.381011:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23714 00000100:00100000:2.0:1787826320.381077:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346226432:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826320.382768:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346226048 00000100:00100000:0.0:1787826320.382786:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346226048 00000100:00100000:0.0:1787826320.382881:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11103 00000100:00100000:0.0:1787826320.382893:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346226048:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826320.382912:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826320.382943:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4ada3 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ad9c expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826320.382966:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7967ac8a00) 00010000:00010000:0.0:1787826320.382981:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826320.383055:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3655:x1874663346226048:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 165us (4973us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826320.383062:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11103 00000100:00100000:1.0:1787826320.383316:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346226048:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826320.384239:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346225920 00000100:00100000:1.0:1787826320.384250:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346225920 00000100:00100000:1.0:1787826320.384304:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11104 00000100:00100000:1.0:1787826320.384314:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd90850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346225920:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826320.384327:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826320.384348:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ad72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x195:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ad6b expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.384366:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x4ec94fd6 (ffff9f784b4a5200) 00010000:00010000:1.0:1787826320.384378:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826320.384428:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd90850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346225920:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 116us (4270us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826320.384436:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11104 00000100:00100000:2.0:1787826320.384559:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346225920:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787826320.386654:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8443:1874663346226560:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826320.386679:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:1.0:1787826320.386696:0:8443:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826320.386755:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332603904 00000100:00100000:2.0:1787826320.386764:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332603904 00010000:00010000:2.0:1787826320.386807:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 7 pid: 35097 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826320.386849:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21957 00000100:00100000:2.0:1787826320.386867:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332603904:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826320.386883:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826320.386890:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 7 pid: 35097 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.386905:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 7 pid: 35097 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.386930:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e23d0662/0xe21309aa81f4ad95 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801552 expref: 7 pid: 35097 timeout: 8232 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.386950:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826320.386955:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826320.386963:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.386973:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826320.387007:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826320.387070:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332603904:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 205us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826320.387079:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21957 00010000:00010000:1.0:1787826320.387837:0:8443:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.387862:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.387874:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.387883:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826320.387902:0:8443:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826320.387956:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826320.388012:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826320.388020:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826320.388025:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826320.388035:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826320.388051:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826320.388058:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009d5fee53/0xe21309aa81f4adbf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826320.388085:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826320.388087:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826320.388088:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826320.388146:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346226176:12345-0@lo:6:osp-syn-1-0.0 Request processed in 7838us (7944us total) trans 4294976159 rc 0/0 00000100:00100000:1.0:1787826320.388156:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30834 00000004:00080000:2.0:1787826320.388309:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784f687800/2, rc 0, transno 4294976159 00000100:00100000:2.0:1787826320.388317:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f687800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346226176:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1787826320.557179:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826320.606041:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826320.606170:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826320.607103:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332604160 00000100:00100000:3.0:1787826320.607154:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332604160 00000100:00100000:3.0:1787826320.607235:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30835 00000100:00100000:3.0:1787826320.607249:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332604160:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:3.0:1787826320.607414:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332604160:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 167us (1389us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826320.607421:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30835 00000100:00100000:3.0:1787826320.607457:0:24220:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332604032 00000100:00100000:3.0:1787826320.607462:0:24220:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332604032 00002000:00100000:3.0:1787826320.607496:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000400:0x4cd:0x0] (0->5) req@ffff9f784f685500 x1874663332604032/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:676/0 lens 496/0 e 0 to 0 dl 1787826331 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826320.607507:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f685500 with x1874663332604032 ext(0->5) 00002000:00010000:3.0:1787826320.607514:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784f685500 00000100:00100000:3.0:1787826320.607544:0:24220:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27649 00000100:00100000:3.0:1787826320.607552:0:24220:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332604032:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826320.610678:0:24220:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332604032:12345-192.168.203.56@tcp:4:bash.0 Request processed in 3107us (4488us total) trans 4294973849 rc 0/0 00000100:00100000:3.0:1787826320.610696:0:24220:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27649 00002000:00100000:3.0:1787826320.610713:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000400:0x4cd:0x0] (0->5) req@ffff9f784f685500 x1874663332604032/t4294973849(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:676/0 lens 496/448 e 0 to 0 dl 1787826331 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826320.610732:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f685500 with x1874663332604032 ext(0->5) 00002000:00010000:3.0:1787826320.610740:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784f685500 00000100:00100000:0.0:1787826320.879978:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346226688:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826320.880063:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826320.880781:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346226688 00000100:00100000:3.0:1787826320.880792:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346226688 00000100:00100000:3.0:1787826320.880859:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6231 00000100:00100000:3.0:1787826320.880872:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346226688:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826320.880907:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914711040 avail 3683188736 00000100:00100000:3.0:1787826320.880992:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346226688:12345-0@lo:13:osp-pre-1-1.0 Request processed in 122us (929us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826320.880999:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6231 00000100:00100000:3.0:1787826320.881058:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346226688:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826321.218683:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.218818:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332604288 00000100:00100000:3.0:1787826321.218831:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332604288 00000100:00100000:3.0:1787826321.218913:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68451 00000100:00100000:3.0:1787826321.218928:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225153:x1874663332604288:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826321.218954:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826321.218981:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4adc6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801560 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.219082:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4adcd lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.219100:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4adcd lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.219139:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4adcd lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826321.219465:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4e:0x0]: returning size 4096 00000004:00200000:3.0:1787826321.219471:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787826321.219495:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4adcd lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.219532:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4adc6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801560 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.219560:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4adcd lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801560 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.219570:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:3.0:1787826321.219574:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826321.219588:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826321.219669:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225153:x1874663332604288:12345-192.168.203.56@tcp:101:rm.0 Request processed in 745us (1000us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.219678:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68451 00000100:00080000:3.0:1787826321.219716:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826321.229999:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.230077:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332604416 00000100:00100000:0.0:1787826321.230090:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332604416 00000100:00100000:0.0:1787826321.230193:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68452 00000100:00100000:0.0:1787826321.230206:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225153:x1874663332604416:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826321.230247:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.230276:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4add4 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801567 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.230357:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.230386:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.230400:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826321.230456:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x195:0x0]: returning size 4096 00000004:00200000:0.0:1787826321.230462:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826321.230504:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4add4 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.230525:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.230536:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:0.0:1787826321.230539:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826321.230544:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826321.230630:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225153:x1874663332604416:12345-192.168.203.56@tcp:101:rm.0 Request processed in 426us (648us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.230638:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68452 00000100:00100000:3.0:1787826321.233175:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.238589:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332604544 00000100:00100000:0.0:1787826321.238613:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332604544 00000100:00100000:0.0:1787826321.238790:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68453 00000100:00100000:0.0:1787826321.238815:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225153:x1874663332604544:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826321.238856:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.238888:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ade2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080156e expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.238972:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.238989:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.239006:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826321.239070:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x193:0x0]: returning size 4096 00000004:00200000:0.0:1787826321.239077:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826321.239153:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ade2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.239178:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.239187:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:0.0:1787826321.239191:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826321.239194:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826321.239288:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225153:x1874663332604544:12345-192.168.203.56@tcp:101:rm.0 Request processed in 478us (6124us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.239297:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68453 00000100:00080000:3.0:1787826321.243611:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826321.245481:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.245574:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332604800 00000100:00100000:1.0:1787826321.245597:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332604800 00000100:00100000:1.0:1787826321.245672:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68454 00000100:00100000:1.0:1787826321.245684:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225153:x1874663332604800:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826321.245712:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826321.245742:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4adf0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801575 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826321.245951:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4e:0x0]: returning size 4096 00010000:00010000:1.0:1787826321.246074:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4adf0 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801575 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.246094:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f794385a100) 00010000:00010000:1.0:1787826321.246097:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826321.246129:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4adf0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801575 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.246150:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826321.246226:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225153:x1874663332604800:12345-192.168.203.56@tcp:101:rm.0 Request processed in 542us (755us total) trans 4294985407 rc 301/301 00000100:00100000:1.0:1787826321.246233:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68454 00000100:00100000:0.0:1787826321.254508:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.254612:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332604928 00000100:00100000:3.0:1787826321.254625:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332604928 00000100:00100000:3.0:1787826321.254702:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9348 00000100:00100000:3.0:1787826321.254714:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225153:x1874663332604928:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826321.260679:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225153:x1874663332604928:12345-192.168.203.56@tcp:37:rm.0 Request processed in 5951us (6170us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.260696:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9348 00000100:00100000:3.0:1787826321.277488:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.277984:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332605056 00000100:00100000:0.0:1787826321.277996:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332605056 00000100:00100000:0.0:1787826321.278073:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9349 00000100:00100000:0.0:1787826321.278086:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225153:x1874663332605056:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826321.290966:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225153:x1874663332605056:12345-192.168.203.56@tcp:37:rm.0 Request processed in 12870us (13482us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.290982:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9349 00000100:00100000:3.0:1787826321.302427:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.304158:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332605312 00000100:00100000:1.0:1787826321.304176:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332605312 00000100:00100000:1.0:1787826321.304261:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9350 00000100:00100000:1.0:1787826321.304276:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cfd1c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225153:x1874663332605312:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:1.0:1787826321.304619:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1203, 344 off1095920 00000040:00080000:1.0:1787826321.304682:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1521, 424 off1537864 00000004:00080000:1.0:1787826321.304690:0:6531:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784548a240 with version 6836 00000004:00080000:3.0:1787826321.305237:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6836 rpc_version 6836 00000020:00100000:3.0:1787826321.305296:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826321.305311:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977dca680 x1874663346226944/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826321.305340:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dca680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346226944:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826321.305431:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.305478:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f797cd23f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.305625:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346226944 00000100:00100000:1.0:1787826321.305634:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346226944 00000100:00100000:1.0:1787826321.305693:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23715 00000100:00100000:1.0:1787826321.305704:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346226944:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826321.305916:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346226944:12345-0@lo:1000:osp_up0-1.0 Request processed in 212us (483us total) trans 8589957400 rc 0/0 00000100:00100000:1.0:1787826321.305928:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23715 00000100:00100000:3.0:1787826321.306406:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957397 gen 2 00000004:00080000:3.0:1787826321.306414:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957396 committed transno 8589957397 00000004:00080000:3.0:1787826321.306441:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957397 committed transno 8589957397 00000100:00100000:3.0:1787826321.306457:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f7977dca680 x1874663346226944/t8589957400(8589957400) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826337 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826321.306483:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dca680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346226944:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826321.306672:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cfd1c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225153:x1874663332605312:12345-192.168.203.56@tcp:35:rm.0 Request processed in 2396us (4254us total) trans 4294985408 rc 0/0 00000100:00100000:1.0:1787826321.306682:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9350 00000100:00100000:1.0:1787826321.314855:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.314953:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332605568 00000100:00100000:3.0:1787826321.314966:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332605568 00000100:00100000:3.0:1787826321.316400:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68455 00000100:00100000:3.0:1787826321.316421:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225153:x1874663332605568:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826321.316456:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826321.316482:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4adfe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080157c expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.316686:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ae05 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.316704:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ae05 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.316716:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ae05 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826321.316729:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4e:0x0]: returning size 4096 00010000:00010000:3.0:1787826321.316848:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4adfe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080157c expref: 13 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.316875:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ae05 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080157c expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.316884:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:3.0:1787826321.316887:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1787826321.316891:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826321.316974:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225153:x1874663332605568:12345-192.168.203.56@tcp:101:rm.0 Request processed in 560us (2133us total) trans 4294985409 rc 0/0 00000100:00100000:3.0:1787826321.316983:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68455 00000100:00100000:3.0:1787826321.326181:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.326286:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332605696 00000100:00100000:0.0:1787826321.326299:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332605696 00000100:00100000:0.0:1787826321.326385:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68456 00000100:00100000:0.0:1787826321.326398:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225153:x1874663332605696:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:0.0:1787826321.326465:0:35098:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345647616 00000100:00100000:0.0:1787826321.326557:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225153:x1874663332605696:12345-192.168.203.56@tcp:41:rm.0 Request processed in 162us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.326566:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68456 00000100:00100000:3.0:1787826321.335985:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.336070:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332605824 00000100:00100000:0.0:1787826321.336082:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332605824 00000100:00100000:0.0:1787826321.336186:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68457 00000100:00100000:0.0:1787826321.336201:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225153:x1874663332605824:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826321.336228:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.336255:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ae13 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801583 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336343:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336360:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336373:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336390:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336397:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336404:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336465:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ae28 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336473:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ae28 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336480:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ae28 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826321.336506:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ae28 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336522:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336529:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336535:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336542:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336548:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826321.336552:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336560:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336573:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:0.0:1787826321.336577:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826321.336593:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 3/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336600:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ae1a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336614:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336621:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336627:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336632:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826321.336634:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336640:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336647:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794b57a200) 00010000:00010000:0.0:1787826321.336649:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826321.336652:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336657:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ae21 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336693:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ae13 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801583 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336708:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ae28 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801583 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.336716:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f794b57aa00) 00010000:00010000:0.0:1787826321.336718:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826321.336721:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826321.336793:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225153:x1874663332605824:12345-192.168.203.56@tcp:101:rm.0 Request processed in 595us (822us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.336800:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68457 00000100:00100000:1.0:1787826321.358229:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.358295:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332605952 00000100:00100000:3.0:1787826321.358308:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332605952 00000100:00100000:3.0:1787826321.358388:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68458 00000100:00100000:3.0:1787826321.358401:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225153:x1874663332605952:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826321.358434:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826321.358461:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ae2f lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080158a expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.358538:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4ae36 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.358554:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4ae36 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.358566:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4ae36 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826321.358636:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4f:0x0]: returning size 6 00000004:00200000:3.0:1787826321.358641:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:3.0:1787826321.358685:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4ae2f lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080158a expref: 14 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.358705:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4ae36 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080158a expref: 13 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.358714:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7843326700) 00010000:00010000:3.0:1787826321.358718:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826321.358723:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826321.358798:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225153:x1874663332605952:12345-192.168.203.56@tcp:101:rm.0 Request processed in 399us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.358805:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68458 00000100:00100000:3.0:1787826321.367456:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.367513:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.367639:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332606080 00000100:00100000:1.0:1787826321.367653:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332606080 00000100:00100000:2.0:1787826321.367661:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332606336 00000100:00100000:2.0:1787826321.367665:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332606336 00000100:00100000:1.0:1787826321.367719:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11105 00000100:00100000:2.0:1787826321.367720:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68459 00000100:00100000:1.0:1787826321.367731:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332606080:12345-192.168.203.56@tcp:103:rm.0 00000100:00100000:2.0:1787826321.367731:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225153:x1874663332606336:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826321.367748:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.367778:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4ae28 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801583 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.367787:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.367798:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f794b57aa00) 00010000:00010000:2.0:1787826321.367800:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4ae36 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080158a expref: 13 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.367811:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826321.367811:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7843326700) 00010000:00010000:2.0:1787826321.367818:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826321.367877:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.367884:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332606080:12345-192.168.203.56@tcp:103:rm.0 Request processed in 154us (444us total) trans 0 rc 0/0 00010000:00010000:2.0:1787826321.367887:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.367892:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11105 00010000:00010000:2.0:1787826321.367898:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.367921:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227072:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1787826321.367966:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.368000:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.368028:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346227072 00000100:00100000:2.0:1787826321.368032:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346227072 00000100:00100000:2.0:1787826321.368074:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23716 00000100:00100000:2.0:1787826321.368083:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:17813:x1874663346227072:12345-0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826321.368100:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.368142:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae44 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ae3d expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.368167:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.368178:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae44 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ae3d expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.368205:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.368220:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 9 pid: 35080 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.368231:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346227200:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826321.368286:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826321.378758:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.378848:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332606464 00000100:00100000:1.0:1787826321.378860:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332606464 00010000:00010000:1.0:1787826321.378916:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 10 pid: 35080 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.378982:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21958 00000100:00100000:1.0:1787826321.378995:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332606464:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.379016:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.379035:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 10 pid: 35080 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379053:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 10 pid: 35080 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379084:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:1.0:1787826321.379092:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae44 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ae3d expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379126:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae44 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4ae3d expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379137:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae44 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4ae3d expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.379169:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ade9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x193:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080156e expref: 10 pid: 35080 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379177:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae44 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae3d expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.379188:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346227200:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826321.379193:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346227328:0@lo:105: 00010000:00010000:2.0:1787826321.379223:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae44 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae3d expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.379233:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:2.0:1787826321.379236:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826321.379239:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:1.0:1787826321.379265:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.379498:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.379518:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346227328 00000100:00100000:1.0:1787826321.379524:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346227328 00000100:00100000:1.0:1787826321.379569:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7869 00000100:00100000:1.0:1787826321.379587:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346227328:12345-0@lo:105: 00010000:00010000:1.0:1787826321.379603:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 4/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379614:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 4/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379623:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 4/0,1 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379632:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379639:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379648:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.379661:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef600) 00000100:00100000:2.0:1787826321.382025:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:17813:x1874663346227072:12345-0@lo:101:mdt00_005.0 Request processed in 13930us (14045us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.382048:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23716 00000100:00100000:2.0:1787826321.382163:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227072:0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826321.382192:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826321.382225:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x193:0x0] 00010000:00010000:2.0:1787826321.382612:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.382623:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.382637:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.382660:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227456:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1787826321.382748:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.382787:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b16c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.382856:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346227456 00000100:00100000:0.0:1787826321.382866:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346227456 00000100:00100000:0.0:1787826321.382928:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23717 00000100:00100000:0.0:1787826321.382939:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:17813:x1874663346227456:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826321.382960:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.382982:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ae52 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ae4b expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.383013:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ae52 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ae4b expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.383028:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ae52 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae4b expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.383048:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794b57a200) 00010000:00010000:0.0:1787826321.383051:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826321.383056:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826321.383181:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346227456:12345-0@lo:101:mdt00_005.0 Request processed in 241us (430us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.383189:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23717 00000100:00100000:0.0:1787826321.383225:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227456:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826321.383238:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.383254:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826321.383508:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x193:0x0] 00000100:00100000:0.0:1787826321.383559:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227584:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826321.383607:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.383628:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.383649:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346227584 00000100:00100000:0.0:1787826321.383654:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346227584 00000100:00100000:0.0:1787826321.383685:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23718 00000100:00100000:0.0:1787826321.383692:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346227584:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826321.383761:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346227584:12345-0@lo:1000:mdt00_005.0 Request processed in 70us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.383768:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23718 00000100:00100000:0.0:1787826321.383800:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227584:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826321.383848:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227712:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826321.383873:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.383886:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.383899:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346227712 00000100:00100000:0.0:1787826321.383902:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346227712 00000100:00100000:0.0:1787826321.383923:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23719 00000100:00100000:0.0:1787826321.383928:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346227712:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826321.383995:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346227712:12345-0@lo:1000:mdt00_005.0 Request processed in 66us (121us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.384002:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23719 00000100:00100000:0.0:1787826321.384028:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227712:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826321.384075:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227840:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826321.384136:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.384150:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.384163:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346227840 00000100:00100000:0.0:1787826321.384166:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346227840 00000100:00100000:0.0:1787826321.384188:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23720 00000100:00100000:0.0:1787826321.384194:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346227840:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826321.384906:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346227840:12345-0@lo:1000:mdt00_005.0 Request processed in 706us (765us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826321.384915:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23720 00000100:00100000:0.0:1787826321.384962:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227840:0@lo:1000:mdt00_005.0 00010000:00010000:0.0:1787826321.385033:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.385052:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.385063:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.385077:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227968:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1787826321.390591:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346227328:12345-0@lo:105: Request processed in 10998us (11307us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.390591:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346228096:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826321.390607:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7869 00000100:00100000:2.0:1787826321.390661:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.390702:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787826321.390708:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346228224:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826321.390728:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826321.390736:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346228608:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826321.390737:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.390765:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.390775:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346228224 00000100:00100000:2.0:1787826321.390783:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346228224 00000100:00100000:1.0:1787826321.390789:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346228352:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826321.390813:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.390827:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346228608 00000100:00100000:2.0:1787826321.390830:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346228608 00000100:00100000:1.0:1787826321.390831:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826321.390835:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346228480:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826321.390859:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.390865:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23721 00000100:00100000:2.0:1787826321.390876:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346228224:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826321.390891:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346228096 00000100:00100000:1.0:1787826321.390894:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346228096 00000100:00100000:1.0:1787826321.390919:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346228352 00000100:00100000:1.0:1787826321.390921:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346228352 00000004:00000024:2.0:1787826321.390922:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442795520 avail 1296556032 00000100:00100000:1.0:1787826321.390931:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346228480 00000100:00100000:1.0:1787826321.390933:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346228480 00000100:00100000:1.0:1787826321.390945:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6232 00000100:00100000:1.0:1787826321.390951:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346228096:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826321.390966:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826321.390971:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346228224:12345-0@lo:41:osp-pre-0-1.0 Request processed in 97us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.390977:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23721 00000100:00100000:2.0:1787826321.390993:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23722 00000100:00100000:1.0:1787826321.390996:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346228096:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (336us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.390999:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346228608:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826321.391002:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6232 00000100:00100000:1.0:1787826321.391013:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6233 00000100:00100000:1.0:1787826321.391018:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346228352:12345-0@lo:13:osp-pre-0-1.0 00000004:00000024:2.0:1787826321.391025:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443401728 avail 1297162240 00000100:00100000:2.0:1787826321.391068:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346228608:12345-0@lo:41:osp-pre-1-0.0 Request processed in 69us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.391075:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23722 00002000:00000024:1.0:1787826321.391133:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826321.391166:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346228352:12345-0@lo:13:osp-pre-0-1.0 Request processed in 148us (352us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.391173:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6233 00000100:00100000:1.0:1787826321.391187:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6234 00000100:00100000:1.0:1787826321.391194:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346228480:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826321.391205:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914711040 avail 3683188736 00000100:00100000:1.0:1787826321.392316:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346228480:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1115us (1449us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.392326:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6234 00000100:00100000:1.0:1787826321.392399:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d24000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346228608:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826321.392437:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346228480:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826321.392451:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346228352:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826321.392482:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346227328:0@lo:105: 00010000:00010000:1.0:1787826321.392497:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.392552:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332606464:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 13558us (13810us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.392560:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21958 00000100:00100000:2.0:1787826321.396661:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346228224:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826321.396695:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346228096:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826321.397965:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.398518:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549fcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.398708:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346227968 00000100:00100000:2.0:1787826321.398721:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346227968 00000100:00100000:2.0:1787826321.398803:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23723 00000100:00100000:2.0:1787826321.398814:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346227968:12345-0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826321.398837:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.398965:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae60 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ae59 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.399010:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae60 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ae59 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.399026:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae60 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae59 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.400029:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7967ac8e00) 00010000:00010000:2.0:1787826321.400228:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826321.400234:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826321.401423:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:17813:x1874663346227968:12345-0@lo:101:mdt00_005.0 Request processed in 2602us (3460us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.401436:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23723 00000100:00100000:2.0:1787826321.401531:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346227968:0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826321.401562:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.401589:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.401641:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.401653:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.401662:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787826321.405572:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].227, 64 off22720 00000004:00080000:2.0:1787826321.405603:0:17813:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8133617496599ns, next at 8134617493096ns, rc = 0 00000040:00080000:2.0:1787826321.405646:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1204, 384 off1096304 00000040:00080000:2.0:1787826321.405720:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1522, 464 off1538328 00000004:00080000:2.0:1787826321.405727:0:17813:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d9b1900 with version 6837 00000004:00080000:2.0:1787826321.405754:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6837 rpc_version 6837 00000020:00100000:2.0:1787826321.405798:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826321.405812:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f796cdb3100 x1874663346228864/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826321.405840:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb3100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346228864:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826321.407466:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.407504:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346228864 00000100:00100000:2.0:1787826321.407514:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b16c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.407516:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346228864 00000100:00100000:1.0:1787826321.407970:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23724 00000100:00100000:1.0:1787826321.407988:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346228864:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826321.409720:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346228864:12345-0@lo:1000:osp_up0-1.0 Request processed in 1731us (2256us total) trans 8589957401 rc 0/0 00000100:00100000:1.0:1787826321.413958:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23724 00000100:00100000:1.0:1787826321.414402:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb3100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346228864:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826321.414464:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414486:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414499:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414509:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414517:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414524:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414544:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826321.414567:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826321.414573:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414592:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826321.414598:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414607:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414621:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7967ac8400) 00010000:00010000:1.0:1787826321.414624:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826321.414630:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414635:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ae6e lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e4f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414648:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.414650:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.414651:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.414666:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414676:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414682:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414686:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826321.414690:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414695:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414703:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7967ac8400) 00010000:00010000:1.0:1787826321.414705:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826321.414708:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414714:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4ae67 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414732:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414740:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414747:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414768:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826321.414787:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x193:0x0] 00010000:00010000:1.0:1787826321.414794:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826321.414799:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414805:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826321.414810:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826321.414817:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414856:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414863:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826321.414889:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414896:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43600/0xe21309aa81f4ae59 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae60 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.414912:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.414914:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.414916:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826321.414932:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346228992:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826321.415011:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.415472:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346228992 00000100:00100000:1.0:1787826321.415482:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346228992 00000100:00100000:1.0:1787826321.415540:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11106 00000100:00100000:1.0:1787826321.415549:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346228992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.415565:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.415599:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae60 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ae59 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415613:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4f:0x0].0x0 (ffff9f7967ac8e00) 00010000:00010000:1.0:1787826321.415621:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.415661:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346228992:12345-0@lo:103:ldlm_bl.0 Request processed in 112us (650us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.415668:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11106 00000100:00100000:1.0:1787826321.415729:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346228992:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.415758:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415766:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415773:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415788:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826321.415794:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x193:0x0] 00010000:00010000:1.0:1787826321.415801:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826321.415807:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415814:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826321.415818:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826321.415825:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415847:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415854:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826321.415882:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 1/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415889:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4ae3d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae44 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.415903:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.415905:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.415907:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826321.415915:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826321.415925:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346229120:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826321.415958:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.415987:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346229120 00000100:00100000:1.0:1787826321.415992:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346229120 00000100:00100000:2.0:1787826321.416193:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11107 00000100:00100000:2.0:1787826321.416205:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346229120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826321.416215:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826321.416230:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae44 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x193:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ae3d expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.416242:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:2.0:1787826321.416251:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826321.416300:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346229120:12345-0@lo:103:ldlm_bl.0 Request processed in 98us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.416307:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11107 00000100:00100000:1.0:1787826321.416829:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346229120:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.416862:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.416872:0:17813:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.416964:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225153:x1874663332606336:12345-192.168.203.56@tcp:36:rm.0 Request processed in 49233us (49448us total) trans 4294985410 rc 0/0 00000100:00100000:1.0:1787826321.416973:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68459 00000100:00100000:2.0:1787826321.422299:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826321.422349:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.422439:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332606592 00000100:00100000:1.0:1787826321.422453:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332606592 00000100:00100000:1.0:1787826321.422537:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11108 00000100:00100000:1.0:1787826321.422551:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332606592:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:1.0:1787826321.422571:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.422615:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4ac8b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801505 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.422642:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:1.0:1787826321.422648:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.422716:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332606592:12345-192.168.203.56@tcp:103:rm.0 Request processed in 168us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.422725:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11108 00000100:00100000:2.0:1787826321.432492:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332606848 00000100:00100000:2.0:1787826321.432514:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332606848 00000100:00100000:2.0:1787826321.432612:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9351 00000100:00100000:2.0:1787826321.432624:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225153:x1874663332606848:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:2.0:1787826321.432936:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:225153:x1874663332606848:12345-192.168.203.56@tcp:35:rm.0 Request processed in 313us (10584us total) trans 4294985411 rc 0/0 00000100:00100000:2.0:1787826321.432944:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9351 00000100:00100000:2.0:1787826321.447801:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.451458:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332607104 00000100:00100000:3.0:1787826321.451478:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332607104 00000100:00100000:3.0:1787826321.451557:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68460 00000100:00100000:3.0:1787826321.451569:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:225153:x1874663332607104:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826321.451653:0:35080:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826321.451688:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4ae05 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080157c expref: 14 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.451712:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4adcd lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801560 expref: 13 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.451724:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4ac99 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080150c expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.451734:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:3.0:1787826321.451740:0:35080:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826321.451799:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.451808:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.451818:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.451843:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0380 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346229248:0@lo:101:mdt00_008.0 00000100:00100000:3.0:1787826321.451922:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.451973:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f794b589600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.452233:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346229248 00000100:00100000:2.0:1787826321.452239:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346229248 00000100:00100000:2.0:1787826321.452292:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23725 00000100:00100000:2.0:1787826321.452301:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35080:x1874663346229248:12345-0@lo:101:mdt00_008.0 00010000:00010000:2.0:1787826321.452317:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.452335:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ae75 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.452361:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.452370:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.452380:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.452408:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.452427:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 7 pid: 17813 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.452439:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346229376:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826321.452523:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.452533:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 7 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.452541:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346229504:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826321.452566:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.456588:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346229376:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826321.456607:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346229504:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826321.456647:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.456662:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826321.456665:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826321.456670:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.456681:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826321.456747:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:35080:x1874663346229248:12345-0@lo:101:mdt00_008.0 Request processed in 4447us (4825us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.456758:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23725 00000100:00100000:3.0:1787826321.456849:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0380 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346229248:0@lo:101:mdt00_008.0 00010000:00010000:3.0:1787826321.456871:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.456883:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.461722:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826321.461810:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.462600:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332607232 00000100:00100000:1.0:1787826321.462616:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332607232 00010000:00010000:1.0:1787826321.462696:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 8 pid: 17813 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.462746:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332607360 00000100:00100000:1.0:1787826321.462750:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332607360 00010000:00010000:1.0:1787826321.462766:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 8 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.462790:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21959 00000100:00100000:1.0:1787826321.462800:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd91450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332607232:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.462819:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.462826:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 8 pid: 17813 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.462841:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 8 pid: 17813 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.462877:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4acd8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801528 expref: 8 pid: 17813 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.462897:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826321.462904:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.462918:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.462986:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd91450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332607232:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 186us (1278us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.462995:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21959 00000100:00100000:1.0:1787826321.463069:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21960 00000100:00100000:1.0:1787826321.463077:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332607360:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.463086:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.463091:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 8 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.463205:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 8 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.463222:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4ac6f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608014fe expref: 8 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.463233:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826321.463237:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.463248:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.463258:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.463653:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.463668:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346229632:0@lo:105: 00000100:00100000:1.0:1787826321.463724:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.463815:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f7d7300 going to sleep for 11 seconds 00000100:00080000:2.0:1787826321.464233:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826321.464727:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346229632 00000100:00100000:0.0:1787826321.464739:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346229632 00000100:00100000:1.0:1787826321.465966:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7870 00000100:00100000:1.0:1787826321.465982:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346229632:12345-0@lo:105: 00010000:00010000:1.0:1787826321.466003:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.466016:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.466026:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.466047:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.466056:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.466067:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.466173:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69600) 00000100:00100000:1.0:1787826321.466284:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346229632:12345-0@lo:105: Request processed in 304us (2558us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.466293:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7870 00000100:00100000:2.0:1787826321.466390:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346229632:0@lo:105: 00010000:00010000:2.0:1787826321.466417:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826321.466474:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332607360:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3395us (4661us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.466483:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21960 00010000:00010000:0.0:1787826321.467599:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.467626:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826321.467659:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826321.467734:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.467742:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.467757:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.467784:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346229760:0@lo:101:mdt00_008.0 00000100:00100000:0.0:1787826321.468202:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.468249:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.468422:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346229760 00000100:00100000:2.0:1787826321.468434:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346229760 00000100:00100000:2.0:1787826321.468507:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23726 00000100:00100000:2.0:1787826321.468519:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35080:x1874663346229760:12345-0@lo:101:mdt00_008.0 00010000:00010000:2.0:1787826321.468542:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.468564:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae8a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ae83 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.468606:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae8a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ae83 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.468620:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae8a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae83 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.468629:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8600) 00010000:00010000:2.0:1787826321.468632:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826321.468636:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826321.468698:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35080:x1874663346229760:12345-0@lo:101:mdt00_008.0 Request processed in 181us (497us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.468706:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23726 00000100:00100000:2.0:1787826321.468752:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75b480 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346229760:0@lo:101:mdt00_008.0 00010000:00010000:2.0:1787826321.468767:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.468776:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826321.468797:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826321.468837:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346229888:0@lo:1000:mdt00_008.0 00000100:00100000:2.0:1787826321.468972:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.468994:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b19c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.469751:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346229888 00000100:00100000:0.0:1787826321.469760:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346229888 00000100:00100000:0.0:1787826321.469807:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23727 00000100:00100000:0.0:1787826321.469814:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:35080:x1874663346229888:12345-0@lo:1000:mdt00_008.0 00000100:00100000:0.0:1787826321.469953:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:35080:x1874663346229888:12345-0@lo:1000:mdt00_008.0 Request processed in 139us (981us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.469958:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23727 00000100:00100000:3.0:1787826321.470017:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346229888:0@lo:1000:mdt00_008.0 00000100:00100000:3.0:1787826321.470438:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd3480 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346230016:0@lo:1000:mdt00_008.0 00000100:00100000:3.0:1787826321.470474:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.470540:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.470569:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346230016 00000100:00100000:2.0:1787826321.470574:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346230016 00000100:00100000:2.0:1787826321.470644:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23728 00000100:00100000:2.0:1787826321.470653:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35080:x1874663346230016:12345-0@lo:1000:mdt00_008.0 00000100:00100000:2.0:1787826321.470781:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35080:x1874663346230016:12345-0@lo:1000:mdt00_008.0 Request processed in 128us (305us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826321.470788:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23728 00000100:00100000:3.0:1787826321.470853:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd3480 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346230016:0@lo:1000:mdt00_008.0 00010000:00010000:3.0:1787826321.470919:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.470930:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.470940:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.470951:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd3480 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346230144:0@lo:101:mdt00_008.0 00000100:00100000:3.0:1787826321.470974:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.471373:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b10c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.471400:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346230144 00000100:00100000:0.0:1787826321.471406:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346230144 00000100:00100000:0.0:1787826321.471448:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23729 00000100:00100000:0.0:1787826321.471457:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35080:x1874663346230144:12345-0@lo:101:mdt00_008.0 00010000:00010000:0.0:1787826321.471472:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.471491:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4ae91 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.471525:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.471535:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4ae91 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.471566:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.471589:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 6 pid: 6529 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.471602:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346230272:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826321.471677:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.475926:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.476080:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346230272:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826321.476171:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae91 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.476186:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826321.476190:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826321.476194:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae91 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.476212:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826321.476304:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:35080:x1874663346230144:12345-0@lo:101:mdt00_008.0 Request processed in 4845us (5326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.476315:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23729 00000100:00100000:1.0:1787826321.476812:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd3480 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346230144:0@lo:101:mdt00_008.0 00010000:00010000:1.0:1787826321.476837:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.476860:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.477598:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332607488 00000100:00100000:3.0:1787826321.477611:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332607488 00010000:00010000:3.0:1787826321.477665:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 7 pid: 6529 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.477744:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21961 00000100:00100000:3.0:1787826321.477757:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332607488:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826321.477775:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826321.477783:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 7 pid: 6529 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.477799:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 7 pid: 6529 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.477826:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4acf4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080152f expref: 7 pid: 6529 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.477843:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:3.0:1787826321.477860:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae91 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.477871:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4ae91 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.477880:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4ae91 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.477920:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae91 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.477935:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346230400:0@lo:105: 00000100:00100000:3.0:1787826321.478007:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.478247:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346230400 00000100:00100000:0.0:1787826321.478254:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346230400 00000100:00100000:0.0:1787826321.478307:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7871 00000100:00100000:0.0:1787826321.478316:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346230400:12345-0@lo:105: 00010000:00010000:0.0:1787826321.478329:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.478340:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.478350:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.478358:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.478365:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.478385:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.478414:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69200) 00000100:00100000:0.0:1787826321.478458:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346230400:12345-0@lo:105: Request processed in 143us (453us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.478466:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7871 00000100:00100000:3.0:1787826321.478738:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b19c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826321.478761:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346230400:0@lo:105: 00010000:00010000:3.0:1787826321.478781:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826321.478838:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332607488:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1083us (2925us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.478853:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21961 00010000:00010000:0.0:1787826321.480484:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480508:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480574:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480600:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480612:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480635:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480649:0:35080:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480658:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480705:0:35080:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.480723:0:35080:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 11 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.480738:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001_UUID:35080:1874663346230528:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826321.480915:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.486178:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826321.486909:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332607616 00000100:00100000:2.0:1787826321.486925:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332607616 00010000:00010000:2.0:1787826321.486979:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 12 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.487040:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21962 00000100:00100000:2.0:1787826321.487053:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332607616:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826321.487075:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826321.487083:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 12 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.487148:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 12 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.487177:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:2.0:1787826321.487183:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.487195:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.487205:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.487229:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826321.487639:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332607616:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 586us (1473us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.487650:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21962 00010000:00010000:0.0:1787826321.489699:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4addb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801567 expref: 11 pid: 8752 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.489731:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001_UUID:35080:1874663346230528:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826321.489769:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.489781:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.489788:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.489951:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 3/0,1 mode: --/EX res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.489960:0:35080:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 3/0,1 mode: --/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.489977:0:35080:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 3/0,1 mode: --/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.490001:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346230656:0@lo:101:mdt00_008.0 00000100:00100000:0.0:1787826321.490370:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.490454:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.490501:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346230656 00000100:00100000:2.0:1787826321.490510:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346230656 00000100:00100000:2.0:1787826321.490611:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23730 00000100:00100000:2.0:1787826321.490623:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:35080:x1874663346230656:12345-0@lo:101:mdt00_008.0 00010000:00010000:2.0:1787826321.490644:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.490665:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aeb4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x193:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4aead expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.490705:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aeb4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aead expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.490721:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aeb4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aead expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.490731:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826321.490735:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826321.490738:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826321.490791:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:35080:x1874663346230656:12345-0@lo:101:mdt00_008.0 Request processed in 170us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.490798:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23730 00000100:00100000:0.0:1787826321.490867:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346230656:0@lo:101:mdt00_008.0 00010000:00010000:0.0:1787826321.490885:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.490897:0:35080:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.491697:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346230784:0@lo:403:mdt00_008.0 00000100:00100000:0.0:1787826321.491778:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.492485:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.492571:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346230784 00000100:00100000:1.0:1787826321.492590:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346230784 00000100:00100000:1.0:1787826321.492687:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23731 00000100:00100000:1.0:1787826321.492698:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35080:x1874663346230784:12345-0@lo:403:mdt00_008.0 00000100:00100000:1.0:1787826321.495096:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35080:x1874663346230784:12345-0@lo:403:mdt00_008.0 Request processed in 2386us (3304us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.495143:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23731 00000100:00100000:1.0:1787826321.497554:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0001-mdtlov_UUID:35080:1874663346230784:0@lo:403:mdt00_008.0 00000040:00080000:1.0:1787826321.500713:0:35080:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1205, 952 off1097256 00000040:00080000:1.0:1787826321.501448:0:35080:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1523, 1032 off1539360 00000004:00080000:1.0:1787826321.501458:0:35080:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784f7d76c0 with version 6838 00000004:00080000:2.0:1787826321.501575:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6838 rpc_version 6838 00000020:00100000:2.0:1787826321.501631:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826321.501644:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d008700 x1874663346231040/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826321.501672:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346231040:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826321.501764:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.501853:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f796cce2f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.501959:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346231040 00000100:00100000:0.0:1787826321.501969:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346231040 00000100:00100000:0.0:1787826321.506321:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23732 00000100:00100000:0.0:1787826321.506356:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346231040:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826321.507444:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346231040:12345-0@lo:1000:osp_up0-1.0 Request processed in 1100us (5672us total) trans 8589957402 rc 0/0 00000100:00100000:0.0:1787826321.507458:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23732 00000100:00100000:2.0:1787826321.507991:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346231040:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787826321.508257:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e4e:0x0]: returning size 4096 00010000:00010000:2.0:1787826321.508296:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508308:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508322:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508330:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826321.508335:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508344:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508364:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x195:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:2.0:1787826321.508368:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826321.508373:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508379:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4aea6 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508403:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508411:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508420:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508441:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826321.508451:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x193:0x0] 00010000:00010000:2.0:1787826321.508461:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826321.508467:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508474:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826321.508478:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826321.508486:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508524:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 2/0,0 mode: --/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508531:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826321.508555:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 1/0,0 mode: --/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508564:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4aead lrc: 0/0,0 mode: --/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aeb4 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508590:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826321.508592:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826321.508593:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826321.508610:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346231168:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826321.508668:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826321.508735:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508744:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508752:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508757:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826321.508761:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508768:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508777:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:2.0:1787826321.508780:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826321.508783:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508790:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4ae9f lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508801:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508810:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508817:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.508838:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826321.508988:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826321.508995:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.509003:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.509010:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.509025:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826321.509029:0:35080:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826321.509045:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.509053:0:35080:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.509549:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225153:x1874663332607104:12345-192.168.203.56@tcp:36:rm.0 Request processed in 57974us (61751us total) trans 4294985412 rc 0/0 00000100:00100000:2.0:1787826321.509560:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68460 00010000:00010000:1.0:1787826321.513166:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826321.513184:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513199:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826321.513207:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826321.513217:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513300:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513307:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826321.513338:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513346:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4ae91 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae98 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513385:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826321.513390:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513396:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826321.513401:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826321.513407:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513424:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513430:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826321.513436:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513448:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4ae75 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae7c expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513456:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.513458:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.513460:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.513468:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826321.513487:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346231424:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826321.513573:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.513636:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826321.513642:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346231296:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826321.513665:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.513699:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346231168 00000100:00100000:1.0:1787826321.513708:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346231168 00000100:00100000:1.0:1787826321.513767:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346231424 00000100:00100000:1.0:1787826321.513770:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346231424 00000100:00100000:1.0:1787826321.513787:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346231296 00000100:00100000:1.0:1787826321.513789:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346231296 00000100:00100000:1.0:1787826321.513807:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11109 00000100:00100000:1.0:1787826321.513817:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+17:3653:x1874663346231168:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.513831:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.513866:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4aeb4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x193:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4aead expref: 17 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.513883:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:1.0:1787826321.513895:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.513937:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+16:3653:x1874663346231168:12345-0@lo:103:ldlm_bl.0 Request processed in 121us (5269us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.513945:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11109 00000100:00100000:1.0:1787826321.513965:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11110 00000100:00100000:1.0:1787826321.513972:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+15:3655:x1874663346231424:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.513981:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.513992:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ae7c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ae75 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.514003:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826321.514006:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.514034:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+14:3655:x1874663346231424:12345-0@lo:103:ldlm_bl.0 Request processed in 64us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.514040:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11110 00000100:00100000:1.0:1787826321.514053:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11111 00000100:00100000:1.0:1787826321.514059:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346231296:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.514067:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.514078:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ae98 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ae91 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.514091:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f784b4a5c00) 00010000:00010000:1.0:1787826321.514099:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.514170:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346231296:12345-0@lo:103:ldlm_bl.0 Request processed in 111us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.514176:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11111 00000100:00100000:1.0:1787826321.514219:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346231296:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826321.514228:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346231424:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826321.514421:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346231168:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.581963:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826321.586246:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.586586:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332607744 00000100:00100000:3.0:1787826321.586599:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332607744 00000100:00100000:3.0:1787826321.586676:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68461 00000100:00100000:3.0:1787826321.586690:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225161:x1874663332607744:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:3.0:1787826321.586739:0:6529:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345647616 00000100:00100000:3.0:1787826321.586836:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225161:x1874663332607744:12345-192.168.203.56@tcp:41:stat.0 Request processed in 148us (604us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.587207:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68461 00000100:00100000:2.0:1787826321.631358:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.631585:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332607872 00000100:00100000:0.0:1787826321.631599:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332607872 00000100:00100000:0.0:1787826321.631678:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68462 00000100:00100000:0.0:1787826321.631691:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225165:x1874663332607872:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826321.631719:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.631745:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4aebb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801591 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.631832:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.631864:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.631878:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826321.631933:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826321.631938:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826321.631960:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.631988:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4aebb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.632012:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.632023:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826321.632026:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826321.632031:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826321.632704:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225165:x1874663332607872:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 1009us (1356us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.632714:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68462 00000100:00100000:0.0:1787826321.636973:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.637074:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332608000 00000100:00100000:3.0:1787826321.637087:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332608000 00000100:00100000:3.0:1787826321.637329:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68463 00000100:00100000:3.0:1787826321.637343:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225165:x1874663332608000:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826321.637385:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826321.637411:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4aec9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801598 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.637492:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.637508:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.637521:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.637538:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.637548:0:35098:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae8a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4ae83 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.637556:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.637611:0:35098:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae8a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4ae83 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.637630:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000_UUID:35098:1874663346231552:0@lo:104: 00000100:00100000:3.0:1787826321.637692:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.637743:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b19c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.637888:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346231552 00000100:00100000:1.0:1787826321.637896:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346231552 00000100:00100000:1.0:1787826321.637942:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7872 00000100:00100000:1.0:1787826321.637951:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35098:x1874663346231552:12345-0@lo:104: 00010000:00010000:1.0:1787826321.637965:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.638042:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826321.638048:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35098:x1874663346231552:12345-0@lo:104: Request processed in 98us (356us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.638056:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7872 00010000:00010000:1.0:1787826321.638080:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826321.638087:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.638128:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826321.638134:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826321.638144:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.638175:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.638182:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826321.638206:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.638211:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42c00/0xe21309aa81f4ae83 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4ae8a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.638233:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.638235:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.638237:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826321.638246:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826321.638258:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346231680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826321.638298:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.638328:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346231680 00000100:00100000:1.0:1787826321.638333:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346231680 00010000:00010000:1.0:1787826321.638354:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae8a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4ae83 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.638596:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000_UUID:35098:1874663346231552:0@lo:104: 00010000:00010000:3.0:1787826321.638619:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.638732:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21963 00000100:00100000:1.0:1787826321.638742:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346231680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.638754:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1787826321.641876:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957402 is committed 00010000:00010000:1.0:1787826321.642196:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ae8a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4ae83 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.642220:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8600) 00010000:00010000:1.0:1787826321.642226:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.642236:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.642249:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.642334:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.642492:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346231680:12345-0@lo:103:ldlm_bl.0 Request processed in 3749us (4191us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.642503:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21963 00010000:00010000:3.0:1787826321.642626:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642636:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642717:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642727:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642735:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642744:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642751:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826321.642757:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642764:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642778:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826321.642781:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826321.642786:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642793:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4aed0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642806:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642812:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642819:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642824:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826321.642827:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642834:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642842:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8600) 00010000:00010000:3.0:1787826321.642857:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826321.642860:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642866:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4aed7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642918:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4aec9 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801598 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642929:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826321.642931:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826321.642936:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4aec9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801598 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.642946:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826321.643014:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225165:x1874663332608000:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 5674us (6055us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826321.643022:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68463 00000100:00100000:2.0:1787826321.644890:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346231680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826321.647406:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.647552:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332608128 00000100:00100000:0.0:1787826321.647565:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332608128 00000100:00100000:0.0:1787826321.647649:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68464 00000100:00100000:0.0:1787826321.647661:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225165:x1874663332608128:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826321.647690:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.647714:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4aede lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080159f expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.647793:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.647811:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.647824:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826321.647875:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826321.647879:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826321.647902:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.647927:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4aede lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.647950:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.647960:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826321.647964:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826321.647969:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826321.648201:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225165:x1874663332608128:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 540us (806us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.648209:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68464 00000100:00100000:3.0:1787826321.651874:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.651949:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332608384 00000100:00100000:1.0:1787826321.651963:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332608384 00000100:00100000:1.0:1787826321.652053:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68465 00000100:00100000:1.0:1787826321.652066:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225165:x1874663332608384:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:1.0:1787826321.652257:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346231808:0@lo:1000:mdt00_012.0 00000100:00080000:2.0:1787826321.652464:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826321.652485:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.652553:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.652613:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346231808 00000100:00100000:2.0:1787826321.652622:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346231808 00000100:00100000:2.0:1787826321.652674:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23733 00000100:00100000:2.0:1787826321.652683:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346231808:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826321.652780:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346231808:12345-0@lo:1000:mdt00_012.0 Request processed in 98us (302us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.652788:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23733 00000100:00100000:2.0:1787826321.652836:0:35095:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957402 gen 2 00000004:00080000:2.0:1787826321.652855:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957400 committed transno 8589957402 00000004:00080000:2.0:1787826321.652882:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957401 committed transno 8589957402 00000004:00080000:2.0:1787826321.652897:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957402 committed transno 8589957402 00000100:00100000:2.0:1787826321.652914:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346231808:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826321.652969:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346231936:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826321.653001:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.653028:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.653532:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346231936 00000100:00100000:1.0:1787826321.653544:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346231936 00000100:00100000:1.0:1787826321.653617:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23734 00000100:00100000:1.0:1787826321.653629:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346231936:12345-0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826321.653791:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346231936:12345-0@lo:1000:mdt00_012.0 Request processed in 164us (789us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826321.653799:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23734 00000100:00100000:2.0:1787826321.653868:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346231936:0@lo:1000:mdt00_012.0 00010000:00010000:2.0:1787826321.653940:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.653952:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.653963:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.653979:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232064:0@lo:101:mdt00_012.0 00000100:00100000:2.0:1787826321.654018:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.654051:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.655759:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346232064 00000100:00100000:1.0:1787826321.655772:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346232064 00000100:00100000:1.0:1787826321.655828:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23735 00000100:00100000:1.0:1787826321.655838:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346232064:12345-0@lo:101:mdt00_012.0 00010000:00010000:1.0:1787826321.655859:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826321.655883:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4aeec expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.655914:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.655935:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.655945:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aeec expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.655979:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.655993:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 7 pid: 6527 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.656007:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346232192:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826321.656164:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.656178:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 6 pid: 35092 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.656189:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346232320:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826321.656237:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.661808:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826321.661958:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346232192:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826321.662266:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332608512 00000100:00100000:3.0:1787826321.662287:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332608512 00010000:00010000:3.0:1787826321.662340:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 8 pid: 6527 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.662390:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21964 00000100:00100000:3.0:1787826321.662402:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332608512:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826321.662418:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826321.662426:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 8 pid: 6527 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.662440:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 8 pid: 6527 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.662466:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4aee5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080159f expref: 8 pid: 6527 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.662483:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826321.662491:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aeec expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.662504:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826321.663041:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332608512:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 635us (1244us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.663051:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21964 00000100:00100000:3.0:1787826321.666894:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.667026:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332608640 00000100:00100000:0.0:1787826321.667038:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332608640 00010000:00010000:0.0:1787826321.667091:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 7 pid: 35092 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.667187:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21965 00000100:00100000:0.0:1787826321.667199:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332608640:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826321.667216:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826321.667224:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 7 pid: 35092 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.667240:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 7 pid: 35092 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.667267:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826321.667275:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aeec expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.667290:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4aeec expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.667300:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4aeec expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.667347:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aeec expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.667365:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346232448:0@lo:105: 00000100:00100000:0.0:1787826321.667428:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.667482:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549f240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826321.667521:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346232448 00000100:00100000:1.0:1787826321.667537:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346232448 00010000:00010000:2.0:1787826321.667614:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4aec2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801591 expref: 7 pid: 35092 timeout: 8233 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.667641:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7873 00000100:00100000:2.0:1787826321.667647:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346232320:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826321.667662:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346232448:12345-0@lo:105: 00010000:00010000:1.0:1787826321.667684:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.667699:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.667701:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aeec expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.667743:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.667746:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826321.667760:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826321.667769:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00010000:00010000:1.0:1787826321.667773:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.667785:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.667794:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.667817:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef000) 00000100:00100000:2.0:1787826321.667838:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346232064:12345-0@lo:101:mdt00_012.0 Request processed in 11999us (13817us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.667865:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23735 00000100:00100000:1.0:1787826321.667913:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346232448:12345-0@lo:105: Request processed in 259us (483us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.667923:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7873 00000100:00100000:2.0:1787826321.667930:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232064:0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826321.667959:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.667983:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346232448:0@lo:105: 00000004:00080000:2.0:1787826321.667987:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826321.668014:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826321.668037:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.668047:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.668056:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.668075:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232576:0@lo:101:mdt00_012.0 00000100:00100000:1.0:1787826321.668086:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332608640:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 884us (1204us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.668098:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21965 00000100:00100000:2.0:1787826321.668168:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.668196:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826321.668872:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346232576 00000100:00100000:3.0:1787826321.668882:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346232576 00000100:00100000:3.0:1787826321.668943:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23736 00000100:00100000:3.0:1787826321.668955:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346232576:12345-0@lo:101:mdt00_012.0 00010000:00010000:3.0:1787826321.668979:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826321.669000:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4af01 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4aefa expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.669029:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4af01 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4aefa expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.669043:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4af01 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aefa expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.669052:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7843326400) 00010000:00010000:3.0:1787826321.669055:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826321.669058:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826321.669146:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346232576:12345-0@lo:101:mdt00_012.0 Request processed in 192us (978us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.669153:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23736 00000100:00100000:2.0:1787826321.669425:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008700 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232576:0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826321.669445:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.669458:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826321.669484:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826321.669535:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232704:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826321.669571:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.669611:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.669647:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346232704 00000100:00100000:0.0:1787826321.669661:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346232704 00000100:00100000:0.0:1787826321.669723:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23737 00000100:00100000:0.0:1787826321.669733:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346232704:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826321.669856:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346232704:12345-0@lo:1000:mdt00_012.0 Request processed in 124us (284us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.669863:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23737 00000100:00100000:3.0:1787826321.670999:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232704:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.671093:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232832:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.671249:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.671289:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.673157:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346232832 00000100:00100000:0.0:1787826321.673173:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346232832 00000100:00100000:0.0:1787826321.673251:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23738 00000100:00100000:0.0:1787826321.673264:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346232832:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826321.673546:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346232832:12345-0@lo:1000:mdt00_012.0 Request processed in 283us (2295us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826321.673557:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23738 00000100:00100000:3.0:1787826321.673621:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232832:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.673780:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232960:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.673917:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.674022:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.677400:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346232960 00000100:00100000:0.0:1787826321.677422:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346232960 00000100:00100000:0.0:1787826321.677508:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23739 00000100:00100000:0.0:1787826321.677522:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346232960:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826321.677740:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346232960:12345-0@lo:1000:mdt00_012.0 Request processed in 221us (3894us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826321.677750:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23739 00000100:00100000:3.0:1787826321.679761:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346232960:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.679884:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346233088:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.679954:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.679994:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.680306:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346233088 00000100:00100000:0.0:1787826321.680315:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346233088 00000100:00100000:0.0:1787826321.680392:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23740 00000100:00100000:0.0:1787826321.680404:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346233088:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826321.680496:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346233088:12345-0@lo:1000:mdt00_012.0 Request processed in 94us (542us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.680504:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23740 00000100:00100000:3.0:1787826321.680595:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346233088:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.680647:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346233216:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.680675:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.680701:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826321.680721:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346233216 00000100:00100000:3.0:1787826321.680726:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346233216 00000100:00100000:3.0:1787826321.680761:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23741 00000100:00100000:3.0:1787826321.680768:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346233216:12345-0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.680839:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346233216:12345-0@lo:1000:mdt00_012.0 Request processed in 71us (163us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.680860:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23741 00000100:00100000:3.0:1787826321.680889:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346233216:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.680929:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346233344:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.680954:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.680968:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826321.680980:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346233344 00000100:00100000:3.0:1787826321.680983:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346233344 00000100:00100000:3.0:1787826321.681000:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23742 00000100:00100000:3.0:1787826321.681006:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346233344:12345-0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826321.681053:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346233344:12345-0@lo:1000:mdt00_012.0 Request processed in 46us (98us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.681060:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23742 00000100:00100000:3.0:1787826321.681085:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346233344:0@lo:1000:mdt00_012.0 00000040:00080000:3.0:1787826321.682010:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1206, 2168 off1099424 00000040:00080000:3.0:1787826321.682142:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1525, 2368 off1542464 00000004:00080000:3.0:1787826321.682150:0:35095:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d9b19c0 with version 6839 00000004:00080000:2.0:1787826321.682379:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6839 rpc_version 6839 00000020:00100000:2.0:1787826321.682421:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826321.682434:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f79465c9c00 x1874663346233600/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826321.682456:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c9c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346233600:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826321.682519:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.682554:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.682595:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346233600 00000100:00100000:2.0:1787826321.682602:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346233600 00000100:00100000:2.0:1787826321.682650:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23743 00000100:00100000:2.0:1787826321.682659:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346233600:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826321.684427:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346233600:12345-0@lo:1000:osp_up0-1.0 Request processed in 1760us (1899us total) trans 8589957403 rc 0/0 00000100:00100000:2.0:1787826321.684441:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23743 00000100:00100000:2.0:1787826321.684601:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c9c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346233600:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826321.689086:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.689197:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.689211:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.689235:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.689243:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.689250:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.689298:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.689306:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.689318:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.689347:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346233728:0@lo:101:mdt00_012.0 00000100:00100000:0.0:1787826321.689432:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.689545:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ff00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.693130:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346233728 00000100:00100000:2.0:1787826321.693153:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346233728 00000100:00100000:2.0:1787826321.693245:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23744 00000100:00100000:2.0:1787826321.693258:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35095:x1874663346233728:12345-0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826321.693434:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.693467:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4af1d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4af16 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.693504:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4af1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4af16 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.693521:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4af1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4af16 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.693531:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:2.0:1787826321.693535:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826321.693540:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826321.693657:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346233728:12345-0@lo:101:mdt00_012.0 Request processed in 402us (4225us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.693664:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23744 00000100:00100000:0.0:1787826321.693974:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346233728:0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826321.694064:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694077:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826321.694188:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000426bce06 reply state 0000000028876c1c transno 4294985413 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694204:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694211:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:0.0:1787826321.694214:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826321.694217:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826321.694224:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694234:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826321.694252:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x194:0x0] 00000004:00010000:0.0:1787826321.694260:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000426bce06 reply state 0000000028876c1c transno 4294985413 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694268:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694275:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826321.694277:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826321.694279:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787826321.694285:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e50:0x0]: returning size 4096 00000004:00080000:0.0:1787826321.694297:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826321.694310:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694316:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694325:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694405:0:35095:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826321.694413:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826321.694418:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.694426:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826321.694535:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225165:x1874663332608384:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 42468us (42683us total) trans 4294985413 rc 0/0 00000100:00100000:0.0:1787826321.694543:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68465 00010000:00010000:1.0:1787826321.699354:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826321.699385:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.699399:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826321.699408:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826321.699418:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.699487:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.699494:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826321.699539:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.699547:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4aeec lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4aef3 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.699568:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.699569:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.699571:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826321.699667:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826321.699695:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346233856:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826321.699786:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.699957:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346233856 00000100:00100000:1.0:1787826321.699968:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346233856 00000100:00100000:1.0:1787826321.700053:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11112 00000100:00100000:1.0:1787826321.700064:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346233856:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.700079:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826321.700151:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4aef3 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4aeec expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.700170:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826321.700176:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826321.700226:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346233856:12345-0@lo:103:ldlm_bl.0 Request processed in 163us (440us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.700233:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11112 00000100:00100000:1.0:1787826321.700285:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346233856:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826321.703766:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.704012:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332608768 00000100:00100000:0.0:1787826321.704026:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332608768 00000100:00100000:0.0:1787826321.704145:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68466 00000100:00100000:0.0:1787826321.704158:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225154:x1874663332608768:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826321.704186:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.704213:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4af24 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015a6 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.704328:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4af2b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.704344:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4af2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.704358:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4af2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826321.704425:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826321.704430:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826321.704454:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4af2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.704483:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4af24 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015a6 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.704511:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4af2b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015a6 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.704521:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826321.704524:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826321.704529:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826321.704620:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225154:x1874663332608768:12345-192.168.203.56@tcp:101:bash.0 Request processed in 463us (865us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.704629:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68466 00000100:00100000:3.0:1787826321.708025:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.711443:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332608896 00000100:00100000:1.0:1787826321.711463:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332608896 00000100:00100000:1.0:1787826321.711563:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68467 00000100:00100000:1.0:1787826321.711588:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225154:x1874663332608896:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826321.711616:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826321.711644:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4af32 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015ad expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.711732:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.711748:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.711761:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.711775:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.711787:0:35094:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4af01 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4aefa expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.711796:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.711841:0:35094:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4af01 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4aefa expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.712310:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000_UUID:35094:1874663346233984:0@lo:104: 00000100:00100000:1.0:1787826321.712683:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.712778:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.714546:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826321.714587:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346233984 00000100:00100000:0.0:1787826321.714598:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346233984 00000100:00100000:0.0:1787826321.714733:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7874 00000100:00100000:0.0:1787826321.714746:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79438e7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:35094:x1874663346233984:12345-0@lo:104: 00010000:00010000:0.0:1787826321.714768:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.714893:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826321.714901:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79438e7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:35094:x1874663346233984:12345-0@lo:104: Request processed in 158us (2219us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.714908:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7874 00010000:00010000:2.0:1787826321.715018:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826321.715028:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.715036:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826321.715043:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826321.715060:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.715102:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.715129:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826321.715156:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.715164:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4aefa lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4af01 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.715188:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826321.715190:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826321.715192:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826321.715210:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d00aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346234112:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826321.715251:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826321.715313:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346234112 00000100:00100000:2.0:1787826321.715328:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346234112 00010000:00010000:2.0:1787826321.715354:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4af01 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4aefa expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.715378:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21966 00000100:00100000:2.0:1787826321.715385:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346234112:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826321.715398:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826321.716903:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000_UUID:35094:1874663346233984:0@lo:104: 00010000:00010000:3.0:1787826321.716938:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826321.718811:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957403 is committed 00010000:00010000:2.0:1787826321.719281:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4af01 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4aefa expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.719309:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7843326400) 00010000:00010000:2.0:1787826321.719316:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.719451:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.719461:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.719611:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826321.719722:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346234112:12345-0@lo:103:ldlm_bl.0 Request processed in 4335us (4468us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.719734:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21966 00010000:00010000:3.0:1787826321.719803:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.719811:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826321.720205:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7480 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-mdtlov_UUID:35094:1874663346234240:0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826321.720230:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d00aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346234112:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826321.720275:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826321.720307:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826321.720432:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346234240 00000100:00100000:2.0:1787826321.720441:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346234240 00000100:00100000:2.0:1787826321.720508:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23745 00000100:00100000:2.0:1787826321.720520:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:35094:x1874663346234240:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826321.720614:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:35094:x1874663346234240:12345-0@lo:1000:mdt00_011.0 Request processed in 96us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.720621:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23745 00000100:00100000:3.0:1787826321.720773:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7480 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-mdtlov_UUID:35094:1874663346234240:0@lo:1000:mdt00_011.0 00010000:00010000:3.0:1787826321.720829:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4af47 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.720854:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4af47 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.720867:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4af47 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826321.721069:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4af47 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721085:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721093:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721099:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721137:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721144:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826321.721149:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721157:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721175:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826321.721178:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826321.721183:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721189:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4af39 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721205:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721211:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721217:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721223:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826321.721226:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721232:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721240:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7843326400) 00010000:00010000:3.0:1787826321.721242:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826321.721245:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721251:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4af40 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721419:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4af32 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015ad expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721438:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4af47 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015ad expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.721457:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f7843326e00) 00010000:00010000:3.0:1787826321.721459:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826321.721462:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826321.721611:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225154:x1874663332608896:12345-192.168.203.56@tcp:101:bash.0 Request processed in 10037us (13601us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.721620:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68467 00000100:00100000:2.0:1787826321.724061:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.724196:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332609024 00000100:00100000:1.0:1787826321.724208:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332609024 00000100:00100000:1.0:1787826321.724275:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68468 00000100:00100000:1.0:1787826321.724286:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225154:x1874663332609024:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826321.724316:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826321.724340:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4af4e lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015b4 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.724414:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4af55 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.724429:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4af55 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.724446:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4af55 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826321.724560:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e50:0x0]: returning size 4096 00000004:00200000:1.0:1787826321.724566:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826321.724614:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4af4e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015b4 expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.724632:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4af55 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015b4 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.724641:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:1.0:1787826321.724644:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826321.724649:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826321.724713:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225154:x1874663332609024:12345-192.168.203.56@tcp:101:bash.0 Request processed in 429us (663us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826321.724719:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68468 00000100:00100000:2.0:1787826321.730486:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.730666:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332609152 00000100:00100000:3.0:1787826321.730679:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332609152 00000100:00100000:3.0:1787826321.730755:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68469 00000100:00100000:3.0:1787826321.730768:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225154:x1874663332609152:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826321.730795:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826321.730819:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4af5c lrc: 2/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015bb expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.730912:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4af63 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.730929:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4af63 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.730942:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4af63 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826321.731013:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x196:0x0]: returning size 4096 00000004:00200000:3.0:1787826321.731020:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826321.731062:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4af5c lrc: 0/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015bb expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.731091:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4af63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015bb expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.731102:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:3.0:1787826321.731145:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826321.731150:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826321.731222:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225154:x1874663332609152:12345-192.168.203.56@tcp:101:bash.0 Request processed in 456us (750us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.731230:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68469 00000100:00100000:3.0:1787826321.742347:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826321.742468:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332609280 00000100:00100000:2.0:1787826321.742480:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332609280 00000100:00100000:2.0:1787826321.742553:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68470 00000100:00100000:2.0:1787826321.742565:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225154:x1874663332609280:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826321.742603:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.742627:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4af6a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015c2 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.742701:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.742719:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4af1d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4af16 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.742730:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.742769:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4af1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4af16 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826321.742786:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6528:1874663346234368:0@lo:104: 00000100:00100000:2.0:1787826321.743468:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826321.743489:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346234368 00000100:00100000:0.0:1787826321.743497:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346234368 00000100:00100000:2.0:1787826321.743515:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826321.743535:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7875 00000100:00100000:0.0:1787826321.743544:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6528:x1874663346234368:12345-0@lo:104: 00010000:00010000:0.0:1787826321.743560:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.743644:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826321.743651:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6528:x1874663346234368:12345-0@lo:104: Request processed in 109us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.743657:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7875 00000100:00100000:2.0:1787826321.743984:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6528:1874663346234368:0@lo:104: 00010000:00010000:2.0:1787826321.744010:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.744642:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826321.744651:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.744661:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826321.744668:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826321.744678:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.747718:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.747740:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826321.747788:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.747798:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4af16 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4af1d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.747837:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.747840:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826321.747843:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826321.748078:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826321.748136:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346234496:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826321.748230:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.748852:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346234496 00000100:00100000:1.0:1787826321.748865:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346234496 00010000:00010000:1.0:1787826321.748913:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4af1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4af16 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.748961:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21967 00000100:00100000:1.0:1787826321.748973:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346234496:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826321.748989:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826321.749601:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4af1d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4af16 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.749623:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:2.0:1787826321.749628:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.749639:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.749646:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.749684:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826321.749763:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346234496:12345-0@lo:103:ldlm_bl.0 Request processed in 792us (1532us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.749772:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21967 00010000:00010000:3.0:1787826321.750389:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.750401:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826321.750480:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x194:0x0]: returning size 4096 00000004:00200000:3.0:1787826321.750487:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826321.750546:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4af6a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015c2 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.750568:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015c2 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.750590:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:3.0:1787826321.750594:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826321.750597:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:3.0:1787826321.750663:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225154:x1874663332609280:12345-192.168.203.56@tcp:101:bash.0 Request processed in 8100us (8331us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.750673:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68470 00000100:00100000:0.0:1787826321.750794:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346234496:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826321.753991:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.754148:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332609536 00000100:00100000:0.0:1787826321.754162:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332609536 00000100:00100000:0.0:1787826321.754235:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68471 00000100:00100000:0.0:1787826321.754250:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd7fc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225154:x1874663332609536:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826321.754280:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.754284:0:24702:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826321.754318:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4af71 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015c2 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.754340:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:0.0:1787826321.754353:0:24702:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826321.754381:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4af78 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015c9 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.754537:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.754557:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.754568:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.754592:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.754600:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.754606:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826321.755511:0:24702:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x520:0x0], index 0: 0 00010000:00010000:0.0:1787826321.755713:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4af8d lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755727:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4af8d lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755736:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4af8d lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755778:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755786:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755793:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755802:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755808:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826321.755813:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755821:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755832:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:0.0:1787826321.755836:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826321.755840:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 3/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755859:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4af7f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826321.755875:0:24702:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002fdef530 reply state 000000001e01e1f6 transno 8589957404 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755888:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755894:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f784b4a5000) 00010000:00010000:0.0:1787826321.755896:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826321.755900:0:24702:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826321.755937:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4af78 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015c9 expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755964:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4af8d lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015c9 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.755972:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826321.755975:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826321.755977:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826321.756226:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd7fc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225154:x1874663332609536:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1977us (2246us total) trans 8589957404 rc 0/0 00000100:00100000:0.0:1787826321.756237:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68471 00000100:00100000:3.0:1787826321.766913:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826321.767971:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332609664 00000100:00100000:0.0:1787826321.767987:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332609664 00000100:00100000:0.0:1787826321.768121:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30836 00000100:00100000:0.0:1787826321.768139:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225154:x1874663332609664:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826321.768165:0:20167:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826321.768194:0:20167:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015d0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826321.768229:0:20167:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x520:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787826321.768268:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=8133s v=(0 0) 00010000:00010000:0.0:1787826321.768274:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826321.768286:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 9 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.768306:0:20167:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 9 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826321.768330:0:20167:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826321.768430:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225154:x1874663332609664:12345-192.168.203.56@tcp:101:bash.0 Request processed in 297us (1531us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826321.768438:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30836 00000100:00080000:3.0:1787826321.770996:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826321.771433:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.771737:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332609920 00000100:00100000:1.0:1787826321.771749:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332609920 00000100:00100000:1.0:1787826321.771824:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9352 00000100:00100000:1.0:1787826321.771837:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225154:x1874663332609920:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:1.0:1787826321.772278:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225154:x1874663332609920:12345-192.168.203.56@tcp:35:bash.0 Request processed in 443us (850us total) trans 8589957405 rc 0/0 00000100:00100000:1.0:1787826321.772290:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9352 00000100:00100000:1.0:1787826321.783768:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826321.783939:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332610176 00000100:00100000:2.0:1787826321.783950:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332610176 00000100:00100000:2.0:1787826321.784091:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68472 00000100:00100000:2.0:1787826321.784145:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225154:x1874663332610176:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826321.784169:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.784428:0:24703:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826321.784457:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4af63 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015bb expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.784478:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:2.0:1787826321.784483:0:24703:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826321.786725:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4afa2 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015d7 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.786933:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.786950:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.786963:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.791731:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.791754:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.791770:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826321.791969:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346234624:0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826321.792046:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826321.792089:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826321.792280:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346234624 00000100:00100000:3.0:1787826321.792287:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346234624 00000100:00100000:3.0:1787826321.793031:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23746 00000100:00100000:3.0:1787826321.793045:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346234624:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826321.793202:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346234624:12345-0@lo:1000:mdt00_007.0 Request processed in 160us (1154us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.793210:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23746 00000100:00100000:1.0:1787826321.793493:0:24703:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957403 gen 2 00000004:00080000:1.0:1787826321.793500:0:24703:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957403 committed transno 8589957403 00000100:00100000:1.0:1787826321.793538:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346234624:0@lo:1000:mdt00_007.0 00000004:00080000:1.0:1787826321.793868:0:24703:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4cb:0x0], index 1: 0 00010000:00010000:1.0:1787826321.794052:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4afb7 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794068:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4afb7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794079:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4afb7 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794146:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794160:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794166:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794230:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794239:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826321.794246:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794253:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794267:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:1.0:1787826321.794270:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826321.794275:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794282:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4afa9 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826321.794298:0:24703:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f570dd0c reply state 000000006bfb64b3 transno 4294985414 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794309:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794452:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x4ec94fd6 (ffff9f7843326400) 00010000:00010000:1.0:1787826321.794456:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826321.794460:0:24703:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826321.794500:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4afa2 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015d7 expref: 12 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794516:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4afb7 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015d7 expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826321.794525:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f794385a100) 00010000:00010000:1.0:1787826321.794527:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826321.794530:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826321.794672:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225154:x1874663332610176:12345-192.168.203.56@tcp:101:bash.0 Request processed in 10569us (10948us total) trans 4294985414 rc 0/0 00000100:00100000:1.0:1787826321.794681:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68472 00000100:00100000:3.0:1787826321.802423:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826321.802603:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332610304 00000100:00100000:2.0:1787826321.802614:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332610304 00000100:00100000:2.0:1787826321.802704:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30837 00000100:00100000:2.0:1787826321.802717:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225154:x1874663332610304:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826321.802736:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826321.802762:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015de expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826321.802790:0:8759:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4cb:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826321.802811:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cb:0x0].0x0 since=8134s v=(0 0) 00010000:00010000:2.0:1787826321.802816:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826321.802826:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 8 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.802843:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 8 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826321.802857:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826321.802933:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225154:x1874663332610304:12345-192.168.203.56@tcp:101:bash.0 Request processed in 219us (520us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826321.802940:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30837 00000100:00080000:3.0:1787826321.803662:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826321.804764:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826321.805136:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332610560 00000100:00100000:2.0:1787826321.805147:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332610560 00000100:00100000:2.0:1787826321.805207:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9353 00000100:00100000:2.0:1787826321.805219:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225154:x1874663332610560:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826321.805522:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225154:x1874663332610560:12345-192.168.203.56@tcp:35:bash.0 Request processed in 304us (764us total) trans 4294985415 rc 0/0 00000100:00100000:2.0:1787826321.805530:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9353 00000100:00100000:1.0:1787826321.826603:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.826652:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826321.826688:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826321.826865:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332610688 00000100:00100000:3.0:1787826321.826875:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332610688 00000100:00100000:3.0:1787826321.826945:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332610816 00000100:00100000:3.0:1787826321.826949:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332610816 00000100:00100000:3.0:1787826321.826970:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332610944 00000100:00100000:3.0:1787826321.826973:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332610944 00000100:00100000:3.0:1787826321.826998:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11113 00000100:00100000:3.0:1787826321.827007:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:34879:x1874663332610688:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826321.827022:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826321.827051:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4af2b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015a6 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.827073:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826321.827084:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4af8d lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015c9 expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.827093:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:3.0:1787826321.827150:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4af47 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015ad expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.827161:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f7843326e00) 00010000:00010000:3.0:1787826321.827168:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826321.827223:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:34879:x1874663332610688:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 217us (630us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.827231:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11113 00000100:00100000:3.0:1787826321.827251:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11114 00000100:00100000:3.0:1787826321.827258:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f5850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:34881:x1874663332610816:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826321.827266:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826321.827277:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4adb1 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dbb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801559 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.827287:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbb:0x0].0x0 (ffff9f794385ae00) 00010000:00010000:3.0:1787826321.827294:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826321.827312:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f5850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:34881:x1874663332610816:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 55us (660us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.827333:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11114 00000100:00100000:3.0:1787826321.827379:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11115 00000100:00100000:3.0:1787826321.827387:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:49150:x1874663332610944:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826321.827395:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826321.827408:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4afb7 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015d7 expref: 13 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.827421:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f794385a100) 00010000:00010000:3.0:1787826321.827433:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4af55 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015b4 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.827441:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:3.0:1787826321.827448:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4ad02 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e4e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801536 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826321.827457:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e4e:0x0].0x0 (ffff9f7843326800) 00010000:00010000:3.0:1787826321.827462:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826321.827479:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:49150:x1874663332610944:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 93us (791us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826321.827486:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11115 00010000:00010000:1.0:1787826322.610343:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826323.212855:0:166246:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:1.0:1787826323.213025:0:166246:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:3.0:1787826323.628374:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826324.652359:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826324.699942:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826324.699985:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826324.700026:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332611072 00000100:00100000:3.0:1787826324.700029:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826324.700038:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332611072 00000100:00100000:0.0:1787826324.700054:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332611328 00000100:00100000:0.0:1787826324.700060:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332611328 00000100:00100000:3.0:1787826324.700067:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826324.700093:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826324.700131:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2226 00000100:00100000:0.0:1787826324.700147:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332611840 00000100:00100000:3.0:1787826324.700148:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826324.700149:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332611072:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826324.700151:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332611840 00000100:00100000:3.0:1787826324.700169:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826324.700175:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332611200 00000100:00100000:0.0:1787826324.700178:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332611200 00000100:00100000:3.0:1787826324.700196:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826324.700198:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14308 00000100:00100000:0.0:1787826324.700207:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2396:x1874663332611328:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826324.700222:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826324.700242:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332611072:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 99us (309us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826324.700251:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2226 00000100:00100000:1.0:1787826324.700284:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332611584 00000100:00100000:1.0:1787826324.700289:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332611584 00000100:00100000:2.0:1787826324.700291:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332611712 00000100:00100000:2.0:1787826324.700294:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332611712 00000100:00100000:1.0:1787826324.700319:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332612096 00000100:00100000:1.0:1787826324.700322:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332612096 00000100:00100000:2.0:1787826324.700322:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14309 00000100:00100000:2.0:1787826324.700328:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2396:x1874663332611840:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826324.700340:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332611456 00000100:00100000:1.0:1787826324.700342:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332611456 00000100:00100000:1.0:1787826324.700361:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332611968 00000100:00100000:1.0:1787826324.700363:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332611968 00000100:00100000:1.0:1787826324.700383:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8825 00000100:00100000:1.0:1787826324.700390:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332611584:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826324.701447:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14310 00000100:00100000:3.0:1787826324.701460:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332611200:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826324.701513:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332611200:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 57us (1367us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826324.701521:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14310 00000100:00100000:3.0:1787826324.701542:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14311 00000100:00100000:3.0:1787826324.701550:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332611712:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826324.701589:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332611712:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 40us (1393us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826324.701596:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14311 00000100:00100000:3.0:1787826324.701622:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8826 00000100:00100000:3.0:1787826324.701630:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332612096:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826324.701657:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332612096:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (1563us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826324.701664:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8826 00000100:00100000:3.0:1787826324.701680:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8827 00000100:00100000:3.0:1787826324.701687:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332611456:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826324.701710:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332611456:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (1542us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826324.701716:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8827 00000100:00100000:3.0:1787826324.701726:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8828 00000100:00100000:3.0:1787826324.701733:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332611968:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826324.701753:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332611968:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (1530us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826324.701759:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8828 00000100:00080000:3.0:1787826324.702155:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826324.702161:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826324.702164:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787826324.702166:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826324.705719:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2396:x1874663332611328:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 5430us (5651us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826324.705737:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14308 00000100:00100000:2.0:1787826324.709092:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2396:x1874663332611840:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 8749us (9009us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826324.709123:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14309 00000100:00100000:1.0:1787826324.715913:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332611584:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 15508us (15869us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826324.715930:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8825 00000100:00100000:2.0:1787826325.485074:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f540a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346234880:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485171:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485226:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f540380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346235392:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485253:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485278:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346235904:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485300:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485325:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:2.0:1787826325.485331:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346234752:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485360:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485379:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346235264:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485403:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485417:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346235776:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485437:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485454:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346236288:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485473:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485498:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346235008:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485522:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485538:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346235520:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485570:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485587:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843090e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346236032:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485603:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485618:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1787826325.485623:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346235136:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485643:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485659:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f540000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346235648:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485677:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485694:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843090700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346236160:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.485714:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.485744:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346235392 00000100:00100000:2.0:1787826325.485750:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346235392 00000100:00100000:2.0:1787826325.485796:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346235776 00000100:00100000:2.0:1787826325.485799:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346235776 00000100:00100000:2.0:1787826325.485816:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346236032 00000100:00100000:0.0:1787826325.485937:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346234880 00000100:00100000:0.0:1787826325.485942:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346234880 00000100:00100000:0.0:1787826325.485973:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346235904 00000100:00100000:0.0:1787826325.485976:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346235904 00000100:00100000:0.0:1787826325.485992:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23747 00000100:00100000:0.0:1787826325.486000:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346234880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826325.486059:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346234880:12345-0@lo:400:kworker.0 Request processed in 60us (889us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826325.486067:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23747 00000100:00100000:0.0:1787826325.486084:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23748 00000100:00100000:0.0:1787826325.486090:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346235904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826325.486135:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346235904:12345-0@lo:400:kworker.0 Request processed in 45us (833us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826325.486141:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23748 00000100:00100000:2.0:1787826325.486921:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346236032 00000100:00100000:2.0:1787826325.486960:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346236160 00000100:00100000:2.0:1787826325.486963:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346236160 00000100:00100000:2.0:1787826325.486990:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8829 00000100:00100000:2.0:1787826325.486999:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346235392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487053:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346235392:12345-0@lo:400:kworker.0 Request processed in 55us (1799us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487063:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8829 00000100:00100000:2.0:1787826325.487078:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8830 00000100:00100000:2.0:1787826325.487084:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346235776:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487126:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346235776:12345-0@lo:400:kworker.0 Request processed in 42us (1688us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487132:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8830 00000100:00100000:2.0:1787826325.487144:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8831 00000100:00100000:2.0:1787826325.487151:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346236032:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487182:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346236032:12345-0@lo:400:kworker.0 Request processed in 32us (1576us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487187:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8831 00000100:00100000:2.0:1787826325.487198:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8832 00000100:00100000:2.0:1787826325.487204:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346236160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487229:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346236160:12345-0@lo:400:kworker.0 Request processed in 26us (1514us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487235:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8832 00000100:00100000:2.0:1787826325.487264:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346235264 00000100:00100000:2.0:1787826325.487268:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346235264 00000100:00100000:2.0:1787826325.487296:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346236288 00000100:00100000:2.0:1787826325.487299:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346236288 00000100:00100000:2.0:1787826325.487314:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346235008 00000100:00100000:2.0:1787826325.487317:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346235008 00000100:00100000:2.0:1787826325.487330:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346235520 00000100:00100000:2.0:1787826325.487333:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346235520 00000100:00100000:2.0:1787826325.487345:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346235136 00000100:00100000:2.0:1787826325.487348:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346235136 00000100:00100000:2.0:1787826325.487361:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346235648 00000100:00100000:2.0:1787826325.487363:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346235648 00000100:00100000:2.0:1787826325.487382:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14312 00000100:00100000:2.0:1787826325.487389:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346235264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487420:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346235264:12345-0@lo:400:kworker.0 Request processed in 32us (2016us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487425:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14312 00000100:00100000:2.0:1787826325.487436:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14313 00000100:00100000:2.0:1787826325.487441:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346236288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487466:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346236288:12345-0@lo:400:kworker.0 Request processed in 25us (1992us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487471:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14313 00000100:00100000:2.0:1787826325.487481:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14314 00000100:00100000:2.0:1787826325.487487:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346235008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487513:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346235008:12345-0@lo:400:kworker.0 Request processed in 27us (1991us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487518:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14314 00000100:00100000:2.0:1787826325.487527:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14315 00000100:00100000:2.0:1787826325.487533:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346235520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487568:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346235520:12345-0@lo:400:kworker.0 Request processed in 36us (1998us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487573:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14315 00000100:00100000:2.0:1787826325.487584:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14316 00000100:00100000:2.0:1787826325.487590:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346235136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487616:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346235136:12345-0@lo:400:kworker.0 Request processed in 27us (1972us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487624:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14316 00000100:00100000:2.0:1787826325.487633:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14317 00000100:00100000:2.0:1787826325.487638:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346235648:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487658:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346235648:12345-0@lo:400:kworker.0 Request processed in 21us (1981us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.487663:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14317 00000100:00100000:2.0:1787826325.487711:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346235264:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487724:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346235776:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487736:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346236288:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487747:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f540a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346234880:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487757:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f540380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346235392:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.487767:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346235904:0@lo:400:kworker.0 00000100:00100000:3.0:1787826325.487982:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346234752 00000100:00100000:3.0:1787826325.487988:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346234752 00000100:00100000:3.0:1787826325.488024:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2227 00000100:00100000:3.0:1787826325.488031:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346234752:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826325.488075:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346234752:12345-0@lo:400:kworker.0 Request processed in 46us (2715us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826325.488081:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2227 00000100:00080000:3.0:1787826325.488143:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787826325.488147:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826325.488150:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826325.488152:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826325.488799:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346234752:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.491266:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346235136:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.491310:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f540000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346235648:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.491326:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843090700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346236160:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.491338:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346235008:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.491349:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346235520:0@lo:400:kworker.0 00000100:00100000:2.0:1787826325.491359:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843090e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346236032:0@lo:400:kworker.0 00010000:00010000:1.0:1787826325.492756:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4ae52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826325.492777:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4ae52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492898:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492907:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492914:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492920:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826325.492924:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492931:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492945:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:3.0:1787826325.492948:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826325.492952:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492959:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4af08 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492981:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492987:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.492993:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493002:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826325.493005:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493011:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493018:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:3.0:1787826325.493020:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826325.493023:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493029:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4af0f lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493051:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493057:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493063:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493067:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826325.493070:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493076:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493087:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x4ec94fd6 (ffff9f7843326400) 00010000:00010000:3.0:1787826325.493089:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826325.493092:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.493097:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4afb0 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.493177:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.493185:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826325.493211:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4ae4b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4ae52 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826325.493228:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985415 is committed 00000020:00080000:1.0:1787826325.493243:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784d9b1480 refcount 2 committed 0 result 0 batchid 3421 00000020:00080000:1.0:1787826325.493246:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d9b19c0 00000020:00080000:1.0:1787826325.493249:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1525 00000020:00080000:1.0:1787826325.493251:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839a00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78491acc00 00000020:00080000:1.0:1787826325.493254:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1206 00000001:00080000:1.0:1787826325.493265:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985412 is committed 00000020:00080000:1.0:1787826325.493269:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784f7d73c0 refcount 2 committed 0 result 0 batchid 3420 00000020:00080000:1.0:1787826325.493271:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed6a700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f7d76c0 00000020:00080000:1.0:1787826325.493276:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1523 00000020:00080000:1.0:1787826325.493278:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed6a000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f796ecd3a00 00000020:00080000:1.0:1787826325.493280:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1205 00000004:00080000:1.0:1787826325.493288:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 365 committed 00000020:00080000:1.0:1787826325.493299:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784d9b1300 refcount 2 committed 0 result 0 batchid 3419 00000020:00080000:1.0:1787826325.493302:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78410c6100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d9b1900 00000020:00080000:1.0:1787826325.493305:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1522 00000020:00080000:1.0:1787826325.493307:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78410c6d00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a191000 00000020:00080000:1.0:1787826325.493310:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1204 00000020:00080000:1.0:1787826325.493319:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784548a300 refcount 2 committed 0 result 0 batchid 3418 00000020:00080000:1.0:1787826325.493321:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc62c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784548a240 00000020:00080000:1.0:1787826325.493323:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1521 00000020:00080000:1.0:1787826325.493325:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc62200 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784b768600 00000020:00080000:1.0:1787826325.493328:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1203 00000001:00080000:1.0:1787826325.493334:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985406 is committed 00000020:00080000:1.0:1787826325.493343:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3417 batchid updated 00000100:00100000:1.0:1787826325.493356:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826325.493374:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346236416:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826325.493446:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:1.0:1787826325.493490:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3421 committed batchid 3417 00000004:00080000:1.0:1787826325.493620:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81180 with version 6840 00000020:00080000:1.0:1787826325.493632:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3414 cancel update log [0x200000401:0x3:0x0].1517: rc = 0 00000020:00080000:1.0:1787826325.493679:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3414 cancel update log [0x240000400:0x3:0x0].1199: rc = 0 00000004:00080000:1.0:1787826325.493703:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e813c0 with version 6841 00000020:00080000:1.0:1787826325.493706:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3415 cancel update log [0x200000401:0x3:0x0].1518: rc = 0 00000020:00080000:1.0:1787826325.493727:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3415 cancel update log [0x240000400:0x3:0x0].1200: rc = 0 00000004:00080000:1.0:1787826325.493743:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e810c0 with version 6842 00000020:00080000:1.0:1787826325.493746:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3416 cancel update log [0x200000401:0x3:0x0].1519: rc = 0 00000020:00080000:1.0:1787826325.493765:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3416 cancel update log [0x240000400:0x3:0x0].1201: rc = 0 00000004:00080000:1.0:1787826325.493783:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81a80 with version 6843 00000020:00080000:1.0:1787826325.493786:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3417 cancel update log [0x200000401:0x3:0x0].1520: rc = 0 00000020:00080000:1.0:1787826325.493804:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3417 cancel update log [0x240000400:0x3:0x0].1202: rc = 0 00000020:00080000:1.0:1787826325.493810:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3417 00000001:00080000:2.0:1787826325.494009:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976159 is committed 00000001:00080000:2.0:1787826325.494017:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976158 is committed 00000001:00080000:2.0:1787826325.494024:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976157 is committed 00000001:00100000:2.0:1787826325.494028:0:9524:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000001:00100000:2.0:1787826325.494032:0:9524:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:2.0:1787826325.494035:0:9524:0:(genops.c:885:class_export_put()) final put ffff9f784407c000/398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:3.0:1787826325.494130:0:128839:0:(genops.c:844:class_export_destroy()) destroying export ffff9f784407c000/398e0287-d3c7-44dd-b1d9-6b85f698e615 for lustre-OST0001 00000020:00000080:3.0:1787826325.494170:0:128839:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9f784407b800 for lustre-OST0001 00000004:00080000:2.0:1787826325.496949:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6840 rpc_version 6840 00000020:00100000:2.0:1787826325.496993:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826325.497003:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977c6c380 x1874663346236672/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826325.497028:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346236672:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826325.497101:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.497168:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826325.497192:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346236416 00000100:00100000:2.0:1787826325.497199:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346236416 00000100:00100000:2.0:1787826325.497256:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11116 00000100:00100000:2.0:1787826325.497266:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346236416:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826325.497280:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826325.497308:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ae52 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x193:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4ae4b expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826325.497323:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x193:0x0].0x5a734ce9 (ffff9f794b57a200) 00010000:00010000:2.0:1787826325.497333:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826325.497372:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346236416:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 107us (3926us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.497379:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11116 00000100:00100000:0.0:1787826325.498178:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346236672 00000100:00100000:0.0:1787826325.498186:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346236672 00000100:00100000:0.0:1787826325.498231:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23749 00000100:00100000:0.0:1787826325.498240:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346236672:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826325.498420:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346236672:12345-0@lo:1000:osp_up0-1.0 Request processed in 180us (1317us total) trans 8589957406 rc 0/0 00000100:00100000:0.0:1787826325.498427:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23749 00000100:00100000:0.0:1787826325.498479:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346236672:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826325.498494:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6841 rpc_version 6841 00000020:00100000:0.0:1787826325.498510:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826325.498516:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784c6b9500 x1874663346236928/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826325.498533:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346236928:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826325.498580:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826325.498599:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b16c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826325.498614:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346236928 00000100:00100000:0.0:1787826325.498618:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346236928 00000100:00100000:0.0:1787826325.498645:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23750 00000100:00100000:0.0:1787826325.498651:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346236928:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826325.498731:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346236928:12345-0@lo:1000:osp_up0-1.0 Request processed in 80us (151us total) trans 8589957407 rc 0/0 00000100:00100000:0.0:1787826325.498738:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23750 00000100:00100000:0.0:1787826325.498771:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346236928:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826325.498778:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6842 rpc_version 6842 00000020:00100000:0.0:1787826325.498789:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826325.498795:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784c6bb800 x1874663346237184/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826325.498809:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346237184:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826325.498952:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826325.498967:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826325.498979:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346237184 00000100:00100000:0.0:1787826325.498982:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346237184 00000100:00100000:0.0:1787826325.499004:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23751 00000100:00100000:0.0:1787826325.499011:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346237184:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826325.499084:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346237184:12345-0@lo:1000:osp_up0-1.0 Request processed in 73us (132us total) trans 8589957408 rc 0/0 00000100:00100000:0.0:1787826325.499091:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23751 00000100:00100000:0.0:1787826325.499950:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346237184:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826325.499960:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6843 rpc_version 6843 00000020:00100000:0.0:1787826325.499974:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826325.499980:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784c6bb480 x1874663346237440/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826325.499994:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346237440:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826325.500027:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826325.500944:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826325.501994:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346236416:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826325.502330:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346237568:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826325.502391:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826325.502440:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346237568 00000100:00100000:1.0:1787826325.502447:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346237568 00000100:00100000:1.0:1787826325.502497:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30838 00000100:00100000:1.0:1787826325.502507:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346237568:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1787826325.502533:0:35097:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1229 count 1 00010000:00010000:1.0:1787826325.502570:0:35097:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.502593:0:35097:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 8 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.502603:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4cd:0x0].0x0 since=10s v=(0 0) 00010000:00010000:1.0:1787826325.502608:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826325.502613:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.502647:0:35097:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 8 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.502663:0:35097:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 8 pid: 20167 timeout: 8237 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826325.502675:0:35097:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:35097:1874663346237696:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826325.502737:0:35097:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826325.504458:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346237440 00000100:00100000:3.0:1787826325.504471:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346237440 00000100:00100000:3.0:1787826325.504575:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23752 00000100:00100000:3.0:1787826325.504586:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346237440:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826325.504793:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346237440:12345-0@lo:1000:osp_up0-1.0 Request processed in 210us (4764us total) trans 8589957409 rc 0/0 00000100:00100000:3.0:1787826325.504801:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23752 00000100:00100000:1.0:1787826325.504888:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346237440:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826325.507695:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826325.508137:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332612224 00000100:00100000:3.0:1787826325.508148:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332612224 00010000:00010000:3.0:1787826325.508199:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 9 pid: 20167 timeout: 8237 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826325.508241:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21968 00000100:00100000:3.0:1787826325.508253:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332612224:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826325.508267:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826325.508274:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 9 pid: 20167 timeout: 8237 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.508288:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 9 pid: 20167 timeout: 8237 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.508313:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826325.508318:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826325.508324:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.508334:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.508356:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826325.508415:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332612224:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 164us (729us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826325.508423:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21968 00010000:00010000:2.0:1787826325.509414:0:35097:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008673aa3a/0xe21309aa81f4ac61 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608014f7 expref: 8 pid: 20167 timeout: 8237 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826325.509438:0:35097:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:35097:1874663346237696:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826325.509462:0:35097:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826325.509801:0:35097:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826325.509808:0:35097:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826325.509818:0:35097:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826325.509835:0:35097:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826325.509858:0:35097:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826325.510097:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826325.510145:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.510154:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826325.510158:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.510166:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.510183:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.510190:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4afcc lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.510210:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826325.510212:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826325.510213:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826325.510432:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346237568:12345-0@lo:6:osp-syn-0-1.0 Request processed in 7924us (8038us total) trans 4294973850 rc 0/0 00000100:00100000:2.0:1787826325.510444:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30838 00000004:00080000:3.0:1787826325.511954:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f79785e4a80/2, rc 0, transno 4294973850 00000100:00100000:3.0:1787826325.511966:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346237568:0@lo:6:osp-syn-0-1.0 00010000:00010000:2.0:1787826325.679947:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826325.724354:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826325.724401:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826325.724483:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332612480 00000100:00100000:2.0:1787826325.724493:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332612480 00000100:00100000:2.0:1787826325.724567:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30839 00000100:00100000:2.0:1787826325.724579:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332612480:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:2.0:1787826325.724689:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332612480:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 112us (286us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.724697:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30839 00000100:00100000:1.0:1787826325.724754:0:20162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332612352 00000100:00100000:1.0:1787826325.724759:0:20162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332612352 00002000:00100000:1.0:1787826325.724793:0:20162:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x520:0x0] (0->5) req@ffff9f7850ad7b80 x1874663332612352/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:681/0 lens 496/0 e 0 to 0 dl 1787826336 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826325.724803:0:20162:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7850ad7b80 with x1874663332612352 ext(0->5) 00002000:00010000:1.0:1787826325.724808:0:20162:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7850ad7b80 00000100:00100000:1.0:1787826325.724826:0:20162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27650 00000100:00100000:1.0:1787826325.724838:0:20162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332612352:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826325.727327:0:20162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2395:x1874663332612352:12345-192.168.203.56@tcp:4:bash.0 Request processed in 2486us (2974us total) trans 4294973851 rc 0/0 00000100:00100000:2.0:1787826325.727341:0:20162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27650 00002000:00100000:2.0:1787826325.727354:0:20162:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x520:0x0] (0->5) req@ffff9f7850ad7b80 x1874663332612352/t4294973851(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:681/0 lens 496/448 e 0 to 0 dl 1787826336 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826325.727377:0:20162:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7850ad7b80 with x1874663332612352 ext(0->5) 00002000:00010000:2.0:1787826325.727384:0:20162:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7850ad7b80 00000100:00100000:0.0:1787826325.731848:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826325.731898:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826325.731903:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332612736 00000100:00100000:2.0:1787826325.731912:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332612736 00000100:00100000:0.0:1787826325.731922:0:24157:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332612608 00000100:00100000:0.0:1787826325.731926:0:24157:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332612608 00002000:00100000:0.0:1787826325.731962:0:24157:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000400:0x4cb:0x0] (0->5) req@ffff9f79465c9c00 x1874663332612608/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:681/0 lens 496/0 e 0 to 0 dl 1787826336 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826325.731974:0:24157:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f79465c9c00 with x1874663332612608 ext(0->5) 00000100:00100000:2.0:1787826325.731976:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30840 00002000:00010000:0.0:1787826325.731979:0:24157:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f79465c9c00 00000100:00100000:2.0:1787826325.731985:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332612736:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:0.0:1787826325.731994:0:24157:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27651 00000100:00100000:0.0:1787826325.732000:0:24157:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332612608:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826325.732094:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332612736:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 109us (256us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.732101:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30840 00000001:00080000:2.0:1787826325.732927:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973851 is committed 00000001:00080000:2.0:1787826325.733048:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973850 is committed 00000100:00100000:3.0:1787826325.733237:0:24157:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332612608:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1232us (1334us total) trans 4294976160 rc 0/0 00000100:00100000:3.0:1787826325.733246:0:24157:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27651 00002000:00100000:3.0:1787826325.733256:0:24157:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000400:0x4cb:0x0] (0->5) req@ffff9f79465c9c00 x1874663332612608/t4294976160(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:681/0 lens 496/448 e 0 to 0 dl 1787826336 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826325.733271:0:24157:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f79465c9c00 with x1874663332612608 ext(0->5) 00002000:00010000:3.0:1787826325.733276:0:24157:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f79465c9c00 00000100:00100000:0.0:1787826325.870687:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826325.871419:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332612864 00000100:00100000:3.0:1787826325.871509:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332612864 00000100:00100000:3.0:1787826325.871593:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68473 00000100:00100000:3.0:1787826325.871606:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225335:x1874663332612864:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826325.871627:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826325.871648:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4afd3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015e5 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.871722:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.871737:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.871749:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826325.871800:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826325.871804:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826325.871826:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.871852:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4afd3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.871869:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.871879:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826325.871882:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826325.871886:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826325.871957:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225335:x1874663332612864:12345-192.168.203.56@tcp:101:cat.0 Request processed in 353us (1280us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826325.871965:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68473 00000100:00100000:2.0:1787826325.879544:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826325.884652:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332612992 00000100:00100000:3.0:1787826325.884670:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332612992 00000100:00100000:1.0:1787826325.891624:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68474 00000100:00100000:1.0:1787826325.891642:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225335:x1874663332612992:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826325.891664:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826325.891687:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4afe1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015ec expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891753:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891768:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891779:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891799:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891807:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891813:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891892:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4aff6 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891900:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4aff6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891906:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4aff6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826325.891930:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4aff6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891946:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891952:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891958:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891965:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891975:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826325.891980:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.891988:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892001:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826325.892004:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826325.892009:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892015:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4afe8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892028:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892034:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892040:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892045:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826325.892048:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892054:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892066:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794385a300) 00010000:00010000:1.0:1787826325.892069:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826325.892072:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892077:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4afef lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892132:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4afe1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015ec expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892148:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4aff6 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015ec expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.892155:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f794385a700) 00010000:00010000:1.0:1787826325.892169:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826325.892172:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826325.892251:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225335:x1874663332612992:12345-192.168.203.56@tcp:101:cat.0 Request processed in 613us (12716us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826325.892259:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68474 00000100:00100000:2.0:1787826325.903801:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826325.903885:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332613120 00000100:00100000:3.0:1787826325.903895:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332613120 00000100:00100000:3.0:1787826325.903959:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68475 00000100:00100000:3.0:1787826325.903971:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:225335:x1874663332613120:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826325.903991:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826325.904016:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4affd lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015f3 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.904085:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b004 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.904099:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b004 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.904132:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b004 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826325.904253:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e50:0x0]: returning size 4096 00000004:00200000:3.0:1787826325.904259:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826325.904301:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4affd lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015f3 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.904321:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b004 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015f3 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.904330:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:3.0:1787826325.904333:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826325.904336:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:3.0:1787826325.904402:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225335:x1874663332613120:12345-192.168.203.56@tcp:101:cat.0 Request processed in 433us (611us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826325.904410:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68475 00000100:00100000:3.0:1787826325.909568:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826325.909645:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332613248 00000100:00100000:1.0:1787826325.909656:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332613248 00000100:00100000:1.0:1787826325.909721:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68476 00000100:00100000:1.0:1787826325.909732:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225335:x1874663332613248:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826325.909752:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826325.909776:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b00b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608015fa expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.909848:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.909860:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.909872:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826325.909924:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x196:0x0]: returning size 4096 00000004:00200000:1.0:1787826325.909931:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826325.909969:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b00b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.909985:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826325.909994:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f794385a300) 00010000:00010000:1.0:1787826325.909997:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826325.910001:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826325.910064:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225335:x1874663332613248:12345-192.168.203.56@tcp:101:cat.0 Request processed in 334us (516us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826325.910071:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68476 00000100:00100000:2.0:1787826325.928040:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826325.928166:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332613376 00000100:00100000:3.0:1787826325.928176:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332613376 00000100:00100000:3.0:1787826325.928236:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68477 00000100:00100000:3.0:1787826325.928246:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225335:x1874663332613376:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826325.928269:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826325.928294:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b019 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801601 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.928361:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.928375:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.928385:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826325.928457:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x194:0x0]: returning size 4096 00000004:00200000:3.0:1787826325.928462:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826325.928501:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b019 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.928516:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.928525:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:3.0:1787826325.928528:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826325.928532:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826325.928601:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225335:x1874663332613376:12345-192.168.203.56@tcp:101:cat.0 Request processed in 356us (570us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826325.928607:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68477 00000100:00100000:2.0:1787826325.934734:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826325.934821:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332613632 00000100:00100000:3.0:1787826325.934842:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332613632 00000100:00100000:3.0:1787826325.934911:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68478 00000100:00100000:3.0:1787826325.934922:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225335:x1874663332613632:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826325.934944:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826325.934967:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b027 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801608 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.935048:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.935062:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.935077:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.935089:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.935097:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826325.935122:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:3.0:1787826325.935198:0:11629:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000100:00100000:3.0:1787826325.996939:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346237824:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826325.997024:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826325.997447:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346237824 00000100:00100000:2.0:1787826325.997456:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346237824 00000100:00100000:2.0:1787826325.997508:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6235 00000100:00100000:2.0:1787826325.997517:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346237824:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1787826325.997542:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826325.997606:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346237824:12345-0@lo:13:osp-pre-1-1.0 Request processed in 91us (583us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826325.997613:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6235 00000100:00100000:3.0:1787826325.997784:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976157 gen 2 00000004:00080000:3.0:1787826325.997790:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784d009180, transno 4294976157 00000100:00100000:3.0:1787826325.997832:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346237824:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826326.398925:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826326.399038:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332613760 00000100:00100000:3.0:1787826326.399049:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332613760 00000100:00100000:3.0:1787826326.399174:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68479 00000100:00100000:3.0:1787826326.399187:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225337:x1874663332613760:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826326.399208:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826326.399230:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b03c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080160f expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.399301:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.399316:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.399328:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826326.399373:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826326.399377:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826326.399400:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.399426:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b03c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.399445:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.399453:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826326.399457:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826326.399461:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826326.399529:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225337:x1874663332613760:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 346us (615us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.399535:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68479 00000100:00080000:0.0:1787826326.399619:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826326.411623:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826326.412018:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332613888 00000100:00100000:3.0:1787826326.412029:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332613888 00000100:00100000:3.0:1787826326.412102:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68480 00000100:00100000:3.0:1787826326.412144:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225337:x1874663332613888:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826326.412168:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826326.412194:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b04a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801616 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412269:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412284:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412295:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412310:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412317:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412324:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412394:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412402:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412409:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826326.412432:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412451:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412457:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412463:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412469:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412475:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826326.412480:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412487:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412500:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826326.412502:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826326.412508:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412514:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b051 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412528:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412535:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412540:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412556:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826326.412559:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412564:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412571:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cc900) 00010000:00010000:3.0:1787826326.412573:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826326.412576:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412584:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b058 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412619:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b04a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.412635:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.413455:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f794385a700) 00010000:00010000:3.0:1787826326.413458:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826326.413461:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826326.413551:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225337:x1874663332613888:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1399us (1927us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.413563:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68480 00000100:00100000:2.0:1787826326.418459:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826326.418551:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332614016 00000100:00100000:0.0:1787826326.418561:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332614016 00000100:00100000:0.0:1787826326.418636:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68481 00000100:00100000:0.0:1787826326.418648:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:225337:x1874663332614016:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826326.418671:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826326.418696:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b066 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080161d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.418764:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b06d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.418778:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b06d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.418788:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b06d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826326.418910:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e50:0x0]: returning size 4096 00000004:00200000:0.0:1787826326.418915:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826326.418968:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b066 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080161d expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.418986:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b06d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080161d expref: 6 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.418994:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:0.0:1787826326.418998:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826326.419001:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826326.419066:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225337:x1874663332614016:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 420us (626us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826326.419073:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68481 00000100:00080000:3.0:1787826326.420905:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826326.430992:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826326.431249:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332614144 00000100:00100000:3.0:1787826326.431259:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332614144 00000100:00100000:3.0:1787826326.431324:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68482 00000100:00100000:3.0:1787826326.431394:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225337:x1874663332614144:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826326.431417:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826326.431441:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b074 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801624 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431511:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 3/1,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431525:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431536:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431562:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431569:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431575:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431665:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b089 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431674:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b089 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431681:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b089 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826326.431724:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e51:0x0]: returning size 6 00000004:00200000:3.0:1787826326.431729:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826326.431749:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b089 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431761:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b089 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431770:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431775:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431781:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431787:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431793:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826326.431797:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.431804:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433199:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f794385a300) 00010000:00010000:3.0:1787826326.433202:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826326.433209:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 3/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433215:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4b07b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433235:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433241:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433247:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433253:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826326.433256:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433263:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433271:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x4ec94fd6 (ffff9f79785cc900) 00010000:00010000:3.0:1787826326.433274:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826326.433276:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433283:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4b082 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433408:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b074 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801624 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433425:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b089 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801624 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.433431:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:3.0:1787826326.433433:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826326.433436:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826326.433579:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225337:x1874663332614144:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2188us (2775us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.433588:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68482 00000100:00100000:2.0:1787826326.444531:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826326.444683:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332614272 00000100:00100000:1.0:1787826326.444693:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332614272 00000100:00100000:1.0:1787826326.444825:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68483 00000100:00100000:1.0:1787826326.444836:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225337:x1874663332614272:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826326.444869:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826326.444892:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b090 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080162b expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445265:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445281:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445292:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445304:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445311:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445317:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445422:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b0a5 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445431:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b0a5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445438:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b0a5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826326.445488:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbc:0x0]: returning size 6 00000004:00200000:1.0:1787826326.445494:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826326.445513:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b0a5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445524:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b0a5 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445533:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.445538:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.446759:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.446769:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.446774:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826326.446779:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.446788:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.446894:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:1.0:1787826326.446899:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826326.446904:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.446910:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4b097 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447099:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447142:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447148:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447152:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826326.447156:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447162:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447172:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f784b4a5000) 00010000:00010000:1.0:1787826326.447174:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826326.447178:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447185:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b09e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447227:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b090 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080162b expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447241:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b0a5 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080162b expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.447254:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f794385a200) 00010000:00010000:1.0:1787826326.447256:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826326.447258:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826326.447478:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225337:x1874663332614272:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2641us (2955us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.447487:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68483 00000100:00100000:2.0:1787826326.460729:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826326.460847:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826326.461000:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332614400 00000100:00100000:1.0:1787826326.461020:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332614400 00000100:00100000:1.0:1787826326.461100:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11117 00000100:00100000:1.0:1787826326.461130:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332614400:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:1.0:1787826326.461149:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826326.461178:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b089 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801624 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.461199:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:1.0:1787826326.461210:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826326.461273:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332614400:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 146us (425us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.461280:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11117 00000100:00100000:0.0:1787826326.468260:0:6540:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332614656 00000100:00100000:0.0:1787826326.468276:0:6540:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332614656 00000100:00100000:0.0:1787826326.468491:0:6540:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1231 00000100:00100000:0.0:1787826326.468503:0:6540:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225337:x1874663332614656:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:0.0:1787826326.468572:0:6540:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826326.468603:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b0a5 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080162b expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.468627:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f794385a200) 00010000:00010000:0.0:1787826326.468638:0:6540:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826326.468721:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346237952:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.468822:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826326.468864:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826326.470942:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346237952 00000100:00100000:2.0:1787826326.470956:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346237952 00000100:00100000:2.0:1787826326.471026:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23753 00000100:00100000:2.0:1787826326.471036:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346237952:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.471464:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346237952:12345-0@lo:1000:mdt_io00_000.0 Request processed in 428us (2641us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826326.471472:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23753 00000100:00100000:0.0:1787826326.471633:0:6540:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985406 gen 2 00000004:00080000:0.0:1787826326.471639:0:6540:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985405 committed transno 4294985406 00000020:00080000:0.0:1787826326.471646:0:6540:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7845dd1480 refcount 2 committed 0 result 0 batchid 4294979083 00000020:00080000:0.0:1787826326.471650:0:6540:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839900 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7845dd1240 00000020:00080000:0.0:1787826326.471653:0:6540:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].939 00000020:00080000:0.0:1787826326.471665:0:6540:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851839d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7851815400 00000020:00080000:0.0:1787826326.471668:0:6540:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].936 00000004:00080000:0.0:1787826326.471716:0:6540:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985406 committed transno 4294985406 00000100:00100000:0.0:1787826326.471730:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346237952:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.471822:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238080:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.471862:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826326.471889:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.472031:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346238080 00000100:00100000:3.0:1787826326.472038:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346238080 00000100:00100000:3.0:1787826326.472084:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23754 00000100:00100000:3.0:1787826326.472092:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238080:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826326.472188:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238080:12345-0@lo:1000:mdt_io00_000.0 Request processed in 97us (326us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.472195:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23754 00000100:00100000:0.0:1787826326.472450:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238080:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826326.472478:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x196:0x0] 00000100:00100000:0.0:1787826326.472509:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238208:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.472539:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826326.472575:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000020:00080000:2.0:1787826326.472999:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979083 committed batchid 4294979082 00000020:00080000:2.0:1787826326.474707:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979082 00000100:00100000:3.0:1787826326.475353:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346238208 00000100:00100000:3.0:1787826326.475364:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346238208 00000100:00100000:3.0:1787826326.475428:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23755 00000100:00100000:3.0:1787826326.475440:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238208:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826326.475601:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238208:12345-0@lo:1000:mdt_io00_000.0 Request processed in 163us (3059us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.475609:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23755 00000100:00100000:0.0:1787826326.476778:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238208:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826326.476827:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e50:0x0] 00000100:00100000:0.0:1787826326.476890:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238336:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.476949:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826326.476985:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.477002:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346238336 00000100:00100000:3.0:1787826326.477147:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346238336 00000100:00100000:3.0:1787826326.477193:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23756 00000100:00100000:3.0:1787826326.477203:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238336:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826326.477292:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238336:12345-0@lo:1000:mdt_io00_000.0 Request processed in 90us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.477300:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23756 00000100:00100000:0.0:1787826326.477451:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238336:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826326.477491:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x196:0x0] 00000100:00100000:0.0:1787826326.477516:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854625880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238464:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.477557:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826326.477580:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.477631:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346238464 00000100:00100000:3.0:1787826326.477636:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346238464 00000100:00100000:3.0:1787826326.477669:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23757 00000100:00100000:3.0:1787826326.477677:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238464:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826326.477736:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238464:12345-0@lo:1000:mdt_io00_000.0 Request processed in 57us (188us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.477744:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23757 00000100:00100000:0.0:1787826326.477766:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854625880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238464:0@lo:1000:mdt_io00_000.0 00000004:00020000:0.0:1787826326.477797:0:6540:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:0.0:1787826326.477846:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 3/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.477856:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 3/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.477867:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 3/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826326.477880:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854625880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238592:0@lo:101:mdt_io00_000.0 00000100:00100000:0.0:1787826326.477906:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826326.477938:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415f00 going to sleep for 11 seconds 00000004:00020000:3.0:1787826326.477971:0:11629:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4458 00000100:00100000:1.0:1787826326.477993:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346238592 00000100:00100000:1.0:1787826326.477998:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346238592 00000100:00100000:1.0:1787826326.478038:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23758 00000100:00100000:1.0:1787826326.478046:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6540:x1874663346238592:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787826326.478054:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826326.478070:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b0ac expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.478097:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.478129:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 4/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b0ac expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.478155:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.478167:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 6 pid: 6528 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.478178:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346238720:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826326.478181:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.478192:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.478202:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826326.478213:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbc:0x0]: returning size 6 00010000:00010000:3.0:1787826326.478297:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.478305:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.478316:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.478322:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.478685:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00010000:00010000:3.0:1787826326.480288:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826326.480295:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480307:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480325:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:3.0:1787826326.480328:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826326.480333:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 3/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480341:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b02e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480358:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480365:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480372:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480383:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826326.480386:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480392:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480400:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f784b4a5000) 00010000:00010000:3.0:1787826326.480403:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826326.480406:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480411:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b035 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480470:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b027 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480486:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.480492:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826326.480495:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826326.480497:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826326.480989:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225335:x1874663332613632:12345-192.168.203.56@tcp:101:cat.0 Request processed in 546067us (546263us total) trans 8589957410 rc 0/0 00000100:00100000:3.0:1787826326.480998:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68478 00000100:00100000:2.0:1787826326.488348:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826326.488454:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826326.489698:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346238720:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826326.489741:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b0ac expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.489843:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f794385a300) 00010000:00010000:1.0:1787826326.489846:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826326.489851:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b0ac expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.489861:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826326.489966:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346238592:12345-0@lo:101:mdt_io00_000.0 Request processed in 11920us (12058us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.489975:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23758 00000100:00100000:2.0:1787826326.490188:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854625880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346238592:0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826326.490211:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 4/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.490223:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 4/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826326.494214:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332614912 00000100:00100000:0.0:1787826326.494233:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332614912 00000100:00100000:0.0:1787826326.494336:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30841 00000100:00100000:0.0:1787826326.494351:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332614912:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826326.494371:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826326.494391:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058d139e7/0xe21309aa81f4b0c8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801632 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826326.494460:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346238848:192.168.203.56@tcp:106: 00000100:00100000:0.0:1787826326.494574:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415a80 going to sleep for 11 seconds 00000100:00080000:2.0:1787826326.495169:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826326.495768:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332614784 00000100:00100000:3.0:1787826326.495778:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332614784 00010000:00010000:1.0:1787826326.496094:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 7 pid: 6528 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.496152:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21969 00000100:00100000:1.0:1787826326.496162:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332614784:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826326.496176:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826326.496183:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 7 pid: 6528 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.496197:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 7 pid: 6528 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.496217:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b012 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015fa expref: 7 pid: 6528 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.496235:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f794385a300) 00010000:00010000:1.0:1787826326.496241:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b0ac expref: 5 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.496261:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b0ac expref: 5 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.496271:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b0ac expref: 5 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.496312:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b0ac expref: 5 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.496338:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6512:1874663346238976:0@lo:105: 00000100:00100000:1.0:1787826326.496399:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826326.496439:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826326.496595:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346238976 00000100:00100000:0.0:1787826326.496602:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346238976 00000100:00100000:0.0:1787826326.496642:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7876 00000100:00100000:0.0:1787826326.496650:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6512:x1874663346238976:12345-0@lo:105: 00010000:00010000:0.0:1787826326.496663:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 5/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.496674:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 5/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.496683:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 5/0,1 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.496690:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.496697:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.496705:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.496735:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42c00) 00000100:00100000:0.0:1787826326.496784:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6512:x1874663346238976:12345-0@lo:105: Request processed in 134us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826326.496791:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7876 00000100:00100000:1.0:1787826326.497443:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6512:1874663346238976:0@lo:105: 00010000:00010000:1.0:1787826326.497465:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826326.497531:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332614784:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1369us (9193us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.497540:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21969 00010000:00010000:1.0:1787826326.497589:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.497600:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826326.497626:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x196:0x0] 00010000:00010000:1.0:1787826326.497667:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.497675:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.497686:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.497705:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346239104:0@lo:101:mdt_io00_000.0 00000100:00100000:1.0:1787826326.497748:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826326.497774:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00000100:00080000:2.0:1787826326.498257:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826326.498817:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346239104 00000100:00100000:3.0:1787826326.498827:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346239104 00000100:00100000:3.0:1787826326.498887:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23759 00000100:00100000:3.0:1787826326.498897:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346239104:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787826326.498926:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826326.498948:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4b0d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b0cf expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.498975:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4b0d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b0cf expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.498987:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4b0d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b0cf expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.498995:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x4ec94fd6 (ffff9f79785cc000) 00010000:00010000:3.0:1787826326.498998:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826326.499001:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826326.499074:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346239104:12345-0@lo:101:mdt_io00_000.0 Request processed in 179us (1326us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.499080:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23759 00000100:00100000:1.0:1787826326.499211:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346239104:0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787826326.499233:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b0d6 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.499245:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b0d6 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826326.499267:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x196:0x0] 00010000:00010000:1.0:1787826326.499285:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.499298:0:6540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.499306:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 4/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.499331:0:6540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.499344:0:6540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 10 pid: 17813 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.499356:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346239232:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826326.499420:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00002000:00010000:1.0:1787826326.499847:0:17826:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000401:0x520:0x0] updating lvb size: 0 -> 6 00002000:00010000:1.0:1787826326.499852:0:17826:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000401:0x520:0x0] updating lvb mtime: 0 -> 1787826320 00002000:00010000:1.0:1787826326.499855:0:17826:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000401:0x520:0x0] updating lvb atime: 0 -> 1787826320 00002000:00010000:1.0:1787826326.499857:0:17826:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000401:0x520:0x0] updating lvb ctime: 0 -> 1787826320 00002000:00010000:1.0:1787826326.499860:0:17826:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000401:0x520:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:1.0:1787826326.499877:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346238848:192.168.203.56@tcp:106: 00010000:00010000:1.0:1787826326.499905:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000058d139e7/0xe21309aa81f4b0c8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801632 expref: 6 pid: 17826 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.499924:0:17826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058d139e7/0xe21309aa81f4b0c8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801632 expref: 6 pid: 17826 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.499936:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:1.0:1787826326.499962:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332614912:12345-192.168.203.56@tcp:101:cat.0 Request processed in 5617us (11508us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826326.499969:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30841 00000100:00100000:1.0:1787826326.507716:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826326.507762:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826326.507833:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346239232:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826326.507864:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332615040 00000100:00100000:3.0:1787826326.507873:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332615040 00000100:00080000:1.0:1787826326.507932:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787826326.507939:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 11 pid: 17813 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.507981:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21970 00000100:00100000:3.0:1787826326.507991:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332615040:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826326.508005:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826326.508011:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 11 pid: 17813 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.508025:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 11 pid: 17813 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.508048:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b020 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801601 expref: 11 pid: 17813 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.508062:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:3.0:1787826326.508067:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.508077:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.508083:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.508095:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.508100:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826326.508134:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.508141:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.508163:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.508172:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.508177:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332615040:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 186us (470us total) trans 0 rc 0/0 00010000:00010000:2.0:1787826326.508185:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.508190:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21970 00000100:00100000:2.0:1787826326.508504:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346239744:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826326.508534:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346239360:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826326.508570:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826326.508589:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.508619:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.508625:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826326.508633:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346239488:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826326.508658:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826326.508692:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346239616:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826326.508693:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346239744 00000100:00100000:1.0:1787826326.508697:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346239744 00000100:00100000:2.0:1787826326.508719:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346239360 00000100:00100000:3.0:1787826326.508720:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.508727:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346239360 00000100:00100000:1.0:1787826326.508729:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346239616 00000100:00100000:1.0:1787826326.508734:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346239616 00000100:00100000:0.0:1787826326.508812:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332615168 00000100:00100000:0.0:1787826326.508817:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332615168 00000100:00100000:0.0:1787826326.508854:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30842 00000100:00100000:0.0:1787826326.508863:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:225335:x1874663332615168:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826326.508876:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826326.508889:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801639 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.508915:0:8445:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 5 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.508923:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x520:0x0].0x0 since=5s v=(0 0) 00010000:00010000:0.0:1787826326.508927:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826326.508932:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 4/0,0 mode: --/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.508956:0:8445:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 5 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.508969:0:8445:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 5 pid: 20167 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826326.508980:0:8445:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346239872:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826326.509031:0:8445:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826326.509047:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346239488 00000100:00100000:0.0:1787826326.509051:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346239488 00000100:00100000:2.0:1787826326.509075:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6236 00000100:00100000:2.0:1787826326.509086:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346239360:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826326.509092:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23760 00000100:00100000:1.0:1787826326.509127:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346239744:12345-0@lo:1000:mdt_io00_000.0 00002000:00000024:2.0:1787826326.509150:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662397440 00000100:00100000:1.0:1787826326.509230:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346239744:12345-0@lo:1000:mdt_io00_000.0 Request processed in 129us (661us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.509237:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23760 00000100:00100000:1.0:1787826326.509254:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23761 00000100:00100000:1.0:1787826326.509260:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346239616:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826326.509282:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443401728 avail 1297162240 00000100:00100000:2.0:1787826326.509306:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346239360:12345-0@lo:13:osp-pre-0-1.0 Request processed in 220us (715us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.509335:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346239616:12345-0@lo:41:osp-pre-1-0.0 Request processed in 74us (614us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.509341:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23761 00000100:00100000:0.0:1787826326.509502:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6237 00000100:00100000:0.0:1787826326.509514:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346239488:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826326.509514:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6236 00002000:00000024:0.0:1787826326.509532:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826326.509583:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346239488:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (923us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826326.509590:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6237 00000100:00100000:2.0:1787826326.509651:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346239744:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.509711:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346240256:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.509737:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.514694:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.516271:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346240256 00000100:00100000:3.0:1787826326.516285:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346240256 00000100:00100000:3.0:1787826326.516356:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23762 00000100:00100000:3.0:1787826326.516367:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346240256:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826326.516506:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346240256:12345-0@lo:1000:mdt_io00_000.0 Request processed in 141us (6768us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.516513:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23762 00000100:00100000:3.0:1787826326.516567:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346240128:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826326.516608:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826326.516651:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976159 gen 2 00000004:00080000:3.0:1787826326.516656:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784f687800, transno 4294976159 00000100:00100000:3.0:1787826326.516678:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346239488:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826326.516692:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973850 gen 2 00000004:00080000:3.0:1787826326.516695:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f79785e4a80, transno 4294973850 00000100:00100000:3.0:1787826326.516709:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346239360:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826326.516801:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346240128 00000100:00100000:1.0:1787826326.516807:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346240128 00000100:00100000:1.0:1787826326.516836:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23763 00000100:00100000:1.0:1787826326.516843:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346240128:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826326.516869:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442791424 avail 1296551936 00000100:00100000:1.0:1787826326.516916:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346240128:12345-0@lo:41:osp-pre-0-1.0 Request processed in 73us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.516922:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23763 00000100:00100000:3.0:1787826326.516923:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346239616:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826326.516935:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826326.516942:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346240000:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826326.516970:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826326.517012:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf28000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346240128:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826326.517396:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346240000 00000100:00100000:1.0:1787826326.517401:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346240000 00000100:00100000:2.0:1787826326.517406:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346240256:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.517478:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346240384:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826326.517503:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6238 00000100:00100000:2.0:1787826326.517509:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826326.517511:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346240000:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826326.517527:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662397440 00000100:00100000:2.0:1787826326.517538:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826326.517574:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346240000:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (603us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.517581:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6238 00000100:00100000:2.0:1787826326.517591:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346240000:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826326.517625:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346240384 00000100:00100000:0.0:1787826326.517630:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346240384 00000100:00100000:0.0:1787826326.517666:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23764 00000100:00100000:0.0:1787826326.517673:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346240384:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.517782:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346240384:12345-0@lo:1000:mdt_io00_000.0 Request processed in 109us (271us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826326.517788:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23764 00000100:00100000:2.0:1787826326.520718:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346240384:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.520849:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346240512:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.520910:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.520950:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826326.521267:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346240512 00000100:00100000:0.0:1787826326.521276:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346240512 00000100:00100000:0.0:1787826326.521362:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23765 00000100:00100000:0.0:1787826326.521372:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346240512:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.521465:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346240512:12345-0@lo:1000:mdt_io00_000.0 Request processed in 94us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826326.521473:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23765 00000100:00100000:2.0:1787826326.521551:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346240512:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.521640:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346240640:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.521667:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826326.527872:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826326.527902:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332615296 00000100:00100000:1.0:1787826326.527911:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332615296 00010000:00010000:1.0:1787826326.527968:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 6 pid: 20167 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.528012:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21971 00000100:00100000:1.0:1787826326.528022:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332615296:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826326.528037:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826326.528060:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 6 pid: 20167 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.528074:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 6 pid: 20167 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.528127:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826326.528133:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826326.528141:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.528151:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.528208:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.528224:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:17812:1874663346240768:192.168.203.56@tcp:105: 00000100:00100000:1.0:1787826326.528305:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00db40 going to sleep for 11 seconds 00000100:00080000:1.0:1787826326.528321:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787826326.528597:0:8445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e23d0662/0xe21309aa81f4af9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015d0 expref: 6 pid: 20167 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.528614:0:8445:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346239872:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826326.528645:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.528657:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:1.0:1787826326.528687:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225335:x1874663332615168:12345-192.168.203.56@tcp:101:cat.0 Request processed in 19824us (20924us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.528695:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30842 00000100:00100000:2.0:1787826326.530252:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.532672:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346240640 00000100:00100000:3.0:1787826326.532687:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346240640 00000100:00100000:3.0:1787826326.532756:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23766 00000100:00100000:3.0:1787826326.532768:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346240640:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826326.532921:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346240640:12345-0@lo:1000:mdt_io00_000.0 Request processed in 155us (11252us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.532929:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23766 00000100:00100000:2.0:1787826326.534769:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346240640:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826326.534832:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e50:0x0] 00010000:00010000:2.0:1787826326.534945:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.534963:0:6540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.534973:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.535010:0:6540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.535025:0:6540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 9 pid: 11629 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826326.535042:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346240896:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826326.535172:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aaf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826326.542994:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346240896:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826326.543028:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.544305:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826326.544457:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332615424 00000100:00100000:2.0:1787826326.544467:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332615424 00010000:00010000:2.0:1787826326.544510:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 10 pid: 11629 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826326.544575:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21972 00000100:00100000:2.0:1787826326.544585:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332615424:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826326.544600:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826326.544608:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 10 pid: 11629 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544621:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 10 pid: 11629 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544643:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b0ba lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801608 expref: 10 pid: 11629 timeout: 8238 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544659:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:2.0:1787826326.544665:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544673:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544681:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544705:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826326.544753:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332615424:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 169us (455us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826326.544760:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21972 00010000:00010000:2.0:1787826326.544860:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544869:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544923:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544931:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.544940:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826326.544959:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241024:0@lo:101:mdt_io00_000.0 00000100:00100000:2.0:1787826326.544999:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.545024:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aaf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826326.545356:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346241024 00000100:00100000:2.0:1787826326.545362:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346241024 00000100:00100000:2.0:1787826326.545393:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23767 00000100:00100000:2.0:1787826326.545401:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346241024:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826326.545415:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826326.545431:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4b100 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b0f9 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.545453:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4b100 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b0f9 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.545465:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4b100 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b0f9 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.545479:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:2.0:1787826326.545482:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787826326.545484:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826326.545519:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241024:12345-0@lo:101:mdt_io00_000.0 Request processed in 118us (519us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826326.545526:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23767 00000100:00100000:2.0:1787826326.545583:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241024:0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787826326.545598:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b100 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.545609:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b100 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826326.545626:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e51:0x0] 00000100:00100000:2.0:1787826326.545671:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241152:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.545699:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.545723:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346241152 00000100:00100000:2.0:1787826326.545727:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346241152 00000100:00100000:2.0:1787826326.545747:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23768 00000100:00100000:2.0:1787826326.545753:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241152:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.546181:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241152:12345-0@lo:1000:mdt_io00_000.0 Request processed in 426us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826326.546189:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23768 00000100:00100000:2.0:1787826326.546212:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826326.546226:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241152:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826326.546255:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x196:0x0] 00000100:00100000:2.0:1787826326.546283:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241280:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.546308:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.546331:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346241280 00000100:00100000:2.0:1787826326.546335:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346241280 00000100:00100000:2.0:1787826326.546355:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23769 00000100:00100000:2.0:1787826326.546362:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241280:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.546413:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241280:12345-0@lo:1000:mdt_io00_000.0 Request processed in 50us (103us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826326.546419:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23769 00000100:00100000:2.0:1787826326.546434:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826326.546444:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf29880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241280:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826326.546457:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e50:0x0] 00010000:00010000:2.0:1787826326.546486:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.546497:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.546507:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826326.546556:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241408:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.546582:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.546601:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346241408 00000100:00100000:2.0:1787826326.546605:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346241408 00000100:00100000:2.0:1787826326.546628:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23770 00000100:00100000:2.0:1787826326.546634:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241408:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.546681:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241408:12345-0@lo:1000:mdt_io00_000.0 Request processed in 47us (98us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826326.546688:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23770 00000100:00100000:2.0:1787826326.546703:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826326.546714:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241408:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826326.546731:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e50:0x0] 00000100:00100000:2.0:1787826326.546751:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241536:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.546773:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.546801:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826326.546833:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346241536 00000100:00100000:0.0:1787826326.546838:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346241536 00000100:00100000:1.0:1787826326.546952:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:17812:1874663346240768:192.168.203.56@tcp:105: 00010000:00010000:1.0:1787826326.546966:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826326.547004:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332615296:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 18984us (19143us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.547012:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21971 00000100:00100000:0.0:1787826326.551932:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23771 00000100:00100000:0.0:1787826326.551962:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241536:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.552124:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241536:12345-0@lo:1000:mdt_io00_000.0 Request processed in 176us (5349us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826326.552132:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23771 00000100:00100000:2.0:1787826326.552156:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241536:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.552234:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241664:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.552271:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.552301:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.553237:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346241664 00000100:00100000:3.0:1787826326.553257:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346241664 00000100:00100000:1.0:1787826326.553459:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23772 00000100:00100000:1.0:1787826326.553470:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241664:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826326.553594:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241664:12345-0@lo:1000:mdt_io00_000.0 Request processed in 125us (1322us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.553601:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23772 00000100:00100000:2.0:1787826326.553797:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826326.553944:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241664:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826326.553980:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x196:0x0] 00000100:00100000:2.0:1787826326.554034:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241792:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.554068:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.554096:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826326.554241:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346241792 00000100:00100000:1.0:1787826326.554256:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346241792 00000100:00100000:1.0:1787826326.554294:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23773 00000100:00100000:1.0:1787826326.554302:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241792:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826326.554367:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241792:12345-0@lo:1000:mdt_io00_000.0 Request processed in 65us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.554375:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23773 00000100:00100000:2.0:1787826326.554474:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241792:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826326.554492:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e50:0x0] 00000100:00100000:2.0:1787826326.554524:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241920:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.554560:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.554582:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826326.554598:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346241920 00000100:00100000:0.0:1787826326.554603:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346241920 00000100:00100000:1.0:1787826326.554629:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23774 00000100:00100000:1.0:1787826326.554636:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241920:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826326.554697:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346241920:12345-0@lo:1000:mdt_io00_000.0 Request processed in 62us (137us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.554705:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23774 00000100:00100000:2.0:1787826326.554762:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346241920:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.554792:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346242048:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826326.554890:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.554912:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826326.554952:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346242048 00000100:00100000:1.0:1787826326.554957:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346242048 00000100:00100000:1.0:1787826326.554986:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23775 00000100:00100000:1.0:1787826326.554993:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346242048:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826326.555186:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346242048:12345-0@lo:1000:mdt_io00_000.0 Request processed in 192us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.555193:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23775 00000100:00100000:3.0:1787826326.555328:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332615552 00000100:00100000:3.0:1787826326.555374:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332615552 00000100:00100000:3.0:1787826326.555452:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68484 00000100:00100000:3.0:1787826326.555460:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225335:x1874663332615552:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826326.555474:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826326.555495:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b10e lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801640 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.555557:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b10e lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801640 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.555574:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b10e lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801640 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.555594:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826326.555597:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787826326.555600:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:0.0:1787826326.555602:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0e300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346242048:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826326.555655:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225335:x1874663332615552:12345-192.168.203.56@tcp:101:cat.0 Request processed in 196us (1861us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.555662:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68484 00000100:00100000:0.0:1787826326.555682:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346242176:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.555719:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826326.555744:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346242176 00000100:00100000:0.0:1787826326.555748:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346242176 00000100:00100000:0.0:1787826326.555773:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23776 00000100:00100000:0.0:1787826326.555779:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346242176:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826326.555838:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346242176:12345-0@lo:1000:mdt_io00_000.0 Request processed in 58us (118us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826326.555844:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23776 00000100:00100000:0.0:1787826326.555862:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7843415f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826326.555877:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346242176:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826326.556288:0:6540:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1dbc:0x0] is inserted into orphan list, open count = 1 00000040:00080000:0.0:1787826326.556360:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].937, 1240 off881968 00000040:00080000:0.0:1787826326.556416:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].940, 1320 off981272 00000004:00080000:0.0:1787826326.556422:0:6540:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7845dd1240 with version 1828 00000004:00080000:1.0:1787826326.556506:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1828 rpc_version 1828 00000020:00100000:1.0:1787826326.556529:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826326.556537:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f785189ea00 x1874663346242432/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826326.556567:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189ea00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346242432:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826326.556607:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826326.556638:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f79679099c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826326.556696:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346242432 00000100:00100000:0.0:1787826326.556701:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346242432 00000100:00100000:0.0:1787826326.556739:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23777 00000100:00100000:0.0:1787826326.556748:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346242432:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826326.556987:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346242432:12345-0@lo:1000:osp_up1-0.0 Request processed in 239us (378us total) trans 4294985416 rc 0/0 00000100:00100000:0.0:1787826326.556994:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23777 00000100:00100000:1.0:1787826326.557091:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189ea00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346242432:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787826326.557208:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557218:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557227:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557232:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826326.557237:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557244:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557258:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:0.0:1787826326.557261:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826326.557265:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557272:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b0f2 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826326.557290:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e51:0x0] 00010000:00010000:0.0:1787826326.557296:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b100 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557304:0:6540:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b100 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557318:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557324:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557333:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557338:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826326.557341:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557347:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557355:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:0.0:1787826326.557357:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826326.557360:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557366:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b0dd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826326.557384:0:6540:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009352b0e6 reply state 000000007ed7d1a4 transno 8589957411 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557396:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557402:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f784b4a5000) 00010000:00010000:0.0:1787826326.557404:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826326.557407:0:6540:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:0.0:1787826326.557410:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x196:0x0] 00010000:00010000:0.0:1787826326.557414:0:6540:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557426:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557433:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557460:0:6540:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826326.557465:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x196:0x0] 00010000:00010000:0.0:1787826326.557470:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b0d6 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557477:0:6540:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b0d6 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557486:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557492:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557498:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557503:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826326.557507:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557513:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557522:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f784abfe600) 00010000:00010000:0.0:1787826326.557524:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826326.557528:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.557534:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b107 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826326.557626:0:6540:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225337:x1874663332614656:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 89124us (96905us total) trans 8589957411 rc 0/0 00000100:00100000:0.0:1787826326.557633:0:6540:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1231 00010000:00010000:1.0:1787826326.557698:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826326.557705:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.557718:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826326.557723:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826326.557732:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.557761:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 2/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.557767:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826326.557802:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 1/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.557809:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42c00/0xe21309aa81f4b0ac lrc: 0/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b0b3 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.557825:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826326.557826:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826326.557828:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826326.557973:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346242560:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826326.558015:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826326.558086:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346242560 00000100:00100000:1.0:1787826326.558093:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346242560 00000100:00100000:1.0:1787826326.558288:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11118 00000100:00100000:1.0:1787826326.558297:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346242560:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826326.558308:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826326.558325:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b0b3 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b0ac expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.558341:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f794385a300) 00010000:00010000:1.0:1787826326.558346:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826326.558377:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346242560:12345-0@lo:103:ldlm_bl.0 Request processed in 82us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826326.558383:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11118 00000100:00100000:1.0:1787826326.559710:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346242560:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826326.562624:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826326.562717:0:8450:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332615680 00000100:00100000:0.0:1787826326.562727:0:8450:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332615680 00002000:00100000:0.0:1787826326.562783:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x520:0x0] (0->4095) req@ffff9f7977e0d180 x1874663332615680/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:682/0 lens 488/0 e 0 to 0 dl 1787826337 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826326.562794:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977e0d180 with x1874663332615680 ext(0->4095) 00002000:00010000:0.0:1787826326.562801:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7977e0d180 00000100:00100000:0.0:1787826326.562905:0:8450:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27652 00000100:00100000:0.0:1787826326.562917:0:8450:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332615680:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:0.0:1787826326.563181:0:8450:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332615680:12345-192.168.203.56@tcp:3:cat.0 Request processed in 265us (567us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787826326.563190:0:8450:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27652 00002000:00100000:0.0:1787826326.563196:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x520:0x0] (0->4095) req@ffff9f7977e0d180 x1874663332615680/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:682/0 lens 488/4536 e 0 to 0 dl 1787826337 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826326.563214:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977e0d180 with x1874663332615680 ext(0->4095) 00002000:00010000:0.0:1787826326.563218:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7977e0d180 00000100:00100000:1.0:1787826326.578270:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826326.578391:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332615936 00000100:00100000:0.0:1787826326.578401:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332615936 00000100:00100000:0.0:1787826326.578558:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9354 00000100:00100000:0.0:1787826326.578571:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225335:x1874663332615936:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:0.0:1787826326.579956:0:6530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1dbc:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1787826326.581091:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].214, 64 off21888 00000004:00080000:0.0:1787826326.581148:0:6530:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8138793041414ns, next at 8139793035897ns, rc = 0 00010000:00010000:0.0:1787826326.581234:0:6530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.581249:0:6530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.581260:0:6530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.581268:0:6530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.581274:0:6530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.581280:0:6530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.581358:0:6530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826326.581495:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225335:x1874663332615936:12345-192.168.203.56@tcp:35:cat.0 Request processed in 2923us (3244us total) trans 8589957412 rc 0/0 00000100:00100000:0.0:1787826326.581506:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9354 00010000:00010000:2.0:1787826326.585413:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826326.585896:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.585910:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826326.585915:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.585925:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.585945:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:2.0:1787826326.585948:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826326.585953:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.585959:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b115 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dbc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.585980:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826326.585982:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826326.585984:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826326.702270:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826326.906338:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b0d6 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826326.906359:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b0d6 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.906409:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b0d6 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.906416:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826326.906456:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecea00/0xe21309aa81f4b0cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b0d6 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.906474:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:0.0:1787826326.906479:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b100 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826326.906485:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b100 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.906499:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346242688:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826326.906502:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b100 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.906507:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826326.906512:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ef800/0xe21309aa81f4b0f9 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b100 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826326.906525:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957412 is committed 00000004:00080000:0.0:1787826326.906534:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 213 committed 00000001:00080000:0.0:1787826326.906578:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957411 is committed 00000100:00100000:3.0:1787826326.906582:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:0.0:1787826326.906591:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979083 batchid updated 00000001:00080000:0.0:1787826326.906600:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957409 is committed 00000100:00100000:3.0:1787826326.906621:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346242816:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826326.906628:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.906636:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.906640:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826326.906644:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.906651:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826326.906655:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.906661:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.906670:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346242688 00010000:00010000:0.0:1787826326.906675:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f784b4a5000) 00000100:00100000:3.0:1787826326.906676:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346242688 00010000:00010000:0.0:1787826326.906678:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826326.906682:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.906688:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b0e4 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.906741:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346242816 00000100:00100000:3.0:1787826326.906744:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346242816 00000100:00100000:3.0:1787826326.906769:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11119 00000100:00100000:3.0:1787826326.906777:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346242688:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826326.906790:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826326.906810:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4b0d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x196:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b0cf expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.906821:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x4ec94fd6 (ffff9f79785cc000) 00010000:00010000:3.0:1787826326.906829:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826326.906863:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+8:3655:x1874663346242688:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 86us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.906869:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11119 00000100:00100000:3.0:1787826326.906889:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11120 00000100:00100000:3.0:1787826326.906896:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346242816:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826326.906903:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826326.906913:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4b100 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b0f9 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.906922:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:3.0:1787826326.906928:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826326.906950:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346242816:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 56us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826326.906955:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11120 00000100:00100000:3.0:1787826326.906990:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346242816:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826326.906999:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346242688:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826326.907178:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346242944:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787826326.907212:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826326.907572:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346242944 00000100:00100000:1.0:1787826326.907578:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346242944 00000100:00100000:1.0:1787826326.907624:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30843 00000100:00100000:1.0:1787826326.907633:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346242944:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1787826326.907655:0:35352:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1312 count 1 00010000:00010000:1.0:1787826326.907692:0:35352:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.907714:0:35352:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.907722:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=1s v=(1 0) 00010000:00010000:1.0:1787826326.907727:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826326.907731:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 4/0,1 mode: --/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.907770:0:35352:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826326.907786:0:35352:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 5 pid: 8445 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826326.907796:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346243072:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826326.907868:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846153900 going to sleep for 11 seconds 00010000:00010000:2.0:1787826326.909044:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.909056:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.909064:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.909070:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826326.909075:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.909084:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.909101:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f784b4a5000) 00010000:00010000:2.0:1787826326.909230:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826326.909236:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.909242:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4af86 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787826326.909291:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979083 00000004:00080000:2.0:1787826326.909354:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f78500aa0c0 with version 1829 00000020:00080000:2.0:1787826326.909412:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979083 cancel update log [0x240000404:0x2:0x0].939: rc = 0 00000020:00080000:2.0:1787826326.909502:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979083 cancel update log [0x200000bd0:0x2:0x0].936: rc = 0 00000020:00080000:2.0:1787826326.909513:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979083 00000004:00080000:1.0:1787826326.916793:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1829 rpc_version 1829 00000100:00100000:2.0:1787826326.920056:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346243072:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826326.920089:0:35352:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826326.922735:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000020:00100000:2.0:1787826326.922838:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826326.922851:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977e0f480 x1874663346243328/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826326.922877:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0f480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346243328:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826326.922934:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826326.922973:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aaf00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826326.923202:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346243328 00000100:00100000:1.0:1787826326.923210:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346243328 00000100:00100000:1.0:1787826326.923279:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23778 00000100:00100000:1.0:1787826326.923291:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346243328:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826326.923462:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346243328:12345-0@lo:1000:osp_up1-0.0 Request processed in 173us (527us total) trans 4294985417 rc 0/0 00000100:00100000:1.0:1787826326.923470:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23778 00000100:00100000:2.0:1787826326.923574:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0f480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346243328:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826326.923644:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332616064 00000100:00100000:0.0:1787826326.923650:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332616064 00010000:00010000:2.0:1787826326.923744:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826326.923818:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21973 00000100:00100000:2.0:1787826326.923827:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332616064:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826326.923850:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826326.923858:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.923873:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.923899:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b0eb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x520:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801639 expref: 6 pid: 8445 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.923920:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826326.923925:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x520:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826326.923931:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.923941:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.923974:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826326.925816:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332616064:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 197us (1296us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826326.925842:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21973 00010000:00010000:0.0:1787826326.925952:0:35352:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.925971:0:35352:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.925983:0:35352:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.925991:0:35352:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.926015:0:35352:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826326.926257:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346242944:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18623us (19042us total) trans 4294973852 rc 0/0 00000100:00100000:0.0:1787826326.926266:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30843 00010000:00010000:0.0:1787826326.926289:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826326.926295:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.926302:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826326.926306:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.926315:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.926332:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 1/0,0 mode: --/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.926340:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006782fc13/0xe21309aa81f4b11c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x520:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826326.926363:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826326.926365:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826326.926366:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1787826326.926404:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f785189ed80/2, rc 0, transno 4294973852 00000100:00100000:0.0:1787826326.926411:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346242944:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1787826327.638852:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826327.638985:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332616192 00000100:00100000:1.0:1787826327.638995:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332616192 00000100:00100000:1.0:1787826327.639070:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68485 00000100:00100000:1.0:1787826327.639081:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225373:x1874663332616192:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826327.639121:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826327.639149:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4b123 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801647 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.639219:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b12a lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.639232:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b12a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.639242:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b12a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826327.639342:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e50:0x0]: returning size 4096 00000004:00200000:1.0:1787826327.639347:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787826327.639367:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b12a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.639394:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4b123 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801647 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.639413:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b12a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801647 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.639421:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:1.0:1787826327.639424:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826327.639427:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826327.639485:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225373:x1874663332616192:12345-192.168.203.56@tcp:101:rm.0 Request processed in 406us (640us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826327.639492:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68485 00000100:00100000:0.0:1787826327.643153:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826327.644746:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332616320 00000100:00100000:2.0:1787826327.644761:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332616320 00000100:00100000:2.0:1787826327.644838:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68486 00000100:00100000:2.0:1787826327.644850:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225373:x1874663332616320:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826327.644872:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826327.644898:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4b131 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080164e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.644960:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.644973:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.644988:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826327.645034:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x196:0x0]: returning size 4096 00000004:00200000:2.0:1787826327.645040:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826327.645079:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4b131 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.645098:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.645124:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f784abfe700) 00010000:00010000:2.0:1787826327.645128:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826327.645131:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826327.645197:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225373:x1874663332616320:12345-192.168.203.56@tcp:101:rm.0 Request processed in 349us (2053us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.645204:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68486 00000100:00100000:2.0:1787826327.652979:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.653079:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332616448 00000100:00100000:3.0:1787826327.653090:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332616448 00000100:00100000:3.0:1787826327.653166:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68487 00000100:00100000:3.0:1787826327.653179:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967913480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225373:x1874663332616448:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826327.653203:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826327.653226:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b13f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801655 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.653345:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.653361:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.653373:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826327.653420:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x194:0x0]: returning size 4096 00000004:00200000:2.0:1787826327.653425:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826327.653464:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b13f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.653480:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.653489:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:2.0:1787826327.653492:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826327.653496:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826327.653568:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967913480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225373:x1874663332616448:12345-192.168.203.56@tcp:101:rm.0 Request processed in 392us (596us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.653575:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68487 00000100:00100000:2.0:1787826327.657385:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.657479:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332616704 00000100:00100000:3.0:1787826327.657490:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332616704 00000100:00100000:3.0:1787826327.657569:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68488 00000100:00100000:3.0:1787826327.657580:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967913100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225373:x1874663332616704:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826327.657600:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826327.657622:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b14d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080165c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826327.657773:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e50:0x0]: returning size 4096 00010000:00010000:3.0:1787826327.657880:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b14d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080165c expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.657902:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f79785cc800) 00010000:00010000:3.0:1787826327.657905:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826327.657910:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b14d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080165c expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.657926:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826327.657986:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967913100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225373:x1874663332616704:12345-192.168.203.56@tcp:101:rm.0 Request processed in 408us (611us total) trans 4294985418 rc 301/301 00000100:00100000:3.0:1787826327.657995:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68488 00000100:00100000:1.0:1787826327.667879:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826327.667992:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332616832 00000100:00100000:2.0:1787826327.668002:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332616832 00000100:00100000:2.0:1787826327.668063:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9355 00000100:00100000:2.0:1787826327.668075:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225373:x1874663332616832:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826327.684301:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225373:x1874663332616832:12345-192.168.203.56@tcp:37:rm.0 Request processed in 16215us (16419us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.684315:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9355 00000100:00100000:2.0:1787826327.696717:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826327.697077:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332616960 00000100:00100000:1.0:1787826327.697089:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332616960 00000100:00100000:1.0:1787826327.697179:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9356 00000100:00100000:1.0:1787826327.697190:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943033050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225373:x1874663332616960:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826327.701495:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943033050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225373:x1874663332616960:12345-192.168.203.56@tcp:37:rm.0 Request processed in 4295us (4776us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.701510:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9356 00000100:00100000:2.0:1787826327.715953:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.716042:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332617216 00000100:00100000:3.0:1787826327.716052:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332617216 00000100:00100000:3.0:1787826327.716149:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9357 00000100:00100000:3.0:1787826327.716161:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225373:x1874663332617216:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:3.0:1787826327.716460:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1207, 344 off1099768 00000040:00080000:3.0:1787826327.716515:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1526, 424 off1542888 00000004:00080000:3.0:1787826327.716521:0:17601:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784f58bd80 with version 6844 00000004:00080000:2.0:1787826327.716659:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6844 rpc_version 6844 00000020:00100000:2.0:1787826327.716692:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826327.716702:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7967911880 x1874663346243584/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826327.716734:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967911880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346243584:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826327.716797:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826327.716908:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826327.716958:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346243584 00000100:00100000:1.0:1787826327.716964:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346243584 00000100:00100000:1.0:1787826327.717012:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23779 00000100:00100000:1.0:1787826327.717021:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346243584:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826327.717216:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346243584:12345-0@lo:1000:osp_up0-1.0 Request processed in 196us (418us total) trans 8589957413 rc 0/0 00000100:00100000:1.0:1787826327.717225:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23779 00000100:00100000:3.0:1787826327.717460:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957409 gen 2 00000004:00080000:3.0:1787826327.717465:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957406 committed transno 8589957409 00000004:00080000:3.0:1787826327.717483:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957407 committed transno 8589957409 00000004:00080000:3.0:1787826327.717492:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957408 committed transno 8589957409 00000004:00080000:3.0:1787826327.717500:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957409 committed transno 8589957409 00000100:00100000:3.0:1787826327.717514:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f7967911880 x1874663346243584/t8589957413(8589957413) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826343 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826327.717560:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967911880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346243584:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826327.717698:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225373:x1874663332617216:12345-192.168.203.56@tcp:35:rm.0 Request processed in 1538us (1754us total) trans 4294985419 rc 0/0 00000100:00100000:2.0:1787826327.717706:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9357 00010000:00010000:0.0:1787826327.724416:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826327.725985:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826327.726164:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332617472 00000100:00100000:0.0:1787826327.726175:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332617472 00000100:00100000:0.0:1787826327.726231:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68489 00000100:00100000:0.0:1787826327.726242:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225373:x1874663332617472:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826327.726266:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826327.726290:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4b15b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801663 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.726480:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b162 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.726495:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b162 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.726507:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b162 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826327.726524:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e50:0x0]: returning size 4096 00010000:00010000:0.0:1787826327.726650:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4b15b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801663 expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.726675:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b162 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801663 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.726685:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:0.0:1787826327.726688:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787826327.726692:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826327.726771:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225373:x1874663332617472:12345-192.168.203.56@tcp:101:rm.0 Request processed in 529us (795us total) trans 4294985420 rc 0/0 00000100:00100000:0.0:1787826327.726779:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68489 00000100:00100000:3.0:1787826327.730694:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826327.730797:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332617600 00000100:00100000:1.0:1787826327.730819:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332617600 00000100:00100000:1.0:1787826327.730890:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68490 00000100:00100000:1.0:1787826327.730901:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225373:x1874663332617600:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:1.0:1787826327.730961:0:6528:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345561600 00000100:00100000:1.0:1787826327.731043:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225373:x1874663332617600:12345-192.168.203.56@tcp:41:rm.0 Request processed in 144us (359us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826327.731051:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68490 00000100:00100000:2.0:1787826327.738745:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.738825:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332617728 00000100:00100000:3.0:1787826327.738834:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332617728 00000100:00100000:3.0:1787826327.738891:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68491 00000100:00100000:3.0:1787826327.738901:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967910a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225373:x1874663332617728:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826327.738922:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826327.738944:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b170 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080166a expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739011:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739025:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739037:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739051:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739067:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739074:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739159:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b185 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739167:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b185 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739174:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b185 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826327.739194:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b185 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739207:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739213:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739220:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739226:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739231:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826327.739235:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739242:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739253:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826327.739256:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826327.739260:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739267:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b177 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739277:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739282:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739288:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739293:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826327.739296:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739302:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739308:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f79785cc800) 00010000:00010000:3.0:1787826327.739310:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826327.739312:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739318:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b17e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739344:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b170 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080166a expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739356:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b185 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080166a expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.739363:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f79785ccb00) 00010000:00010000:3.0:1787826327.739366:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826327.739368:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826327.739422:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967910a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225373:x1874663332617728:12345-192.168.203.56@tcp:101:rm.0 Request processed in 523us (687us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.739428:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68491 00000100:00100000:0.0:1787826327.741825:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826327.741911:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332617856 00000100:00100000:2.0:1787826327.741922:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332617856 00000100:00100000:2.0:1787826327.741990:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68492 00000100:00100000:2.0:1787826327.742001:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225373:x1874663332617856:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826327.742027:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826327.742063:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4b18c lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801671 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.742166:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b193 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.742181:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b193 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.742192:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b193 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826327.742242:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e51:0x0]: returning size 6 00000004:00200000:2.0:1787826327.742247:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:2.0:1787826327.742285:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4b18c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801671 expref: 14 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.742309:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b193 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801671 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.742319:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:2.0:1787826327.742322:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826327.742326:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826327.742462:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225373:x1874663332617856:12345-192.168.203.56@tcp:101:rm.0 Request processed in 463us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.742469:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68492 00000100:00100000:3.0:1787826327.762920:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.762961:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.762990:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332617984 00000100:00100000:3.0:1787826327.762999:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332617984 00000100:00100000:2.0:1787826327.763086:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332618240 00000100:00100000:2.0:1787826327.763091:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332618240 00000100:00100000:2.0:1787826327.763147:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68493 00000100:00100000:2.0:1787826327.763158:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225373:x1874663332618240:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:2.0:1787826327.763213:0:24703:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826327.763240:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b193 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801671 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.763258:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f784abfe900) 00010000:00010000:2.0:1787826327.763271:0:24703:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826327.763325:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.763334:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.763343:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.763368:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32d80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346243712:0@lo:101:mdt00_007.0 00000100:00100000:2.0:1787826327.763437:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826327.763474:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.763568:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346243712 00000100:00100000:0.0:1787826327.763572:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346243712 00000100:00100000:0.0:1787826327.763607:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23780 00000100:00100000:0.0:1787826327.763615:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346243712:12345-0@lo:101:mdt00_007.0 00010000:00010000:0.0:1787826327.763629:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826327.763647:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b1a1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b19a expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.763669:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.763678:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b1a1 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b19a expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.763704:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.763717:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 11 pid: 35093 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826327.763728:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346243840:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826327.763791:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453ce540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.768635:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11121 00000100:00100000:3.0:1787826327.768659:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332617984:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826327.768683:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826327.768713:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b185 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080166a expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.768729:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f79785ccb00) 00010000:00010000:3.0:1787826327.768740:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826327.768798:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332617984:12345-192.168.203.56@tcp:103:rm.0 Request processed in 150us (5889us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.768998:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11121 00000100:00100000:3.0:1787826327.770354:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.770404:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332618368 00000100:00100000:3.0:1787826327.770414:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332618368 00010000:00010000:3.0:1787826327.770455:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 10 pid: 35093 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.770489:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21974 00000100:00100000:3.0:1787826327.770499:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332618368:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.770513:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826327.770521:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 10 pid: 35093 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.770911:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 10 pid: 35093 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.770938:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826327.770945:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b1a1 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b19a expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.770957:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b1a1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4b19a expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.770966:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b1a1 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4b19a expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.771000:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b1a1 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b19a expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.771015:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851241180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346243968:0@lo:105: 00000100:00100000:3.0:1787826327.771072:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.771133:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b0c0 going to sleep for 11 seconds 00010000:00010000:3.0:1787826327.771178:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b146 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x194:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801655 expref: 10 pid: 35093 timeout: 8239 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.771196:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346243840:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826327.771223:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b1a1 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b19a expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.771231:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826327.771234:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826327.771236:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:3.0:1787826327.771275:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:24703:x1874663346243712:12345-0@lo:101:mdt00_007.0 Request processed in 7661us (7838us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.771282:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23780 00000100:00100000:3.0:1787826327.771323:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32d80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346243712:0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826327.771338:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 4/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.771349:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 4/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826327.771691:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346243968 00000100:00100000:1.0:1787826327.771697:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346243968 00000100:00100000:1.0:1787826327.772863:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7877 00000100:00100000:1.0:1787826327.772878:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346243968:12345-0@lo:105: 00010000:00010000:1.0:1787826327.772894:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 5/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.772906:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 5/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.772915:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 5/0,1 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.772924:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.772933:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.772942:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.772964:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490efa00) 00000100:00100000:1.0:1787826327.773036:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346243968:12345-0@lo:105: Request processed in 162us (1963us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826327.773043:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7877 00010000:00010000:3.0:1787826327.774928:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.774944:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826327.774979:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x194:0x0] 00010000:00010000:3.0:1787826327.775027:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.775035:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.775047:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.775071:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244096:0@lo:101:mdt00_007.0 00000100:00100000:3.0:1787826327.775147:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.775178:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.775211:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851241180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346243968:0@lo:105: 00010000:00010000:3.0:1787826327.775227:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826327.775279:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332618368:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 4778us (4928us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.775289:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21974 00000100:00100000:3.0:1787826327.775361:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346244096 00000100:00100000:3.0:1787826327.775368:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346244096 00000100:00100000:3.0:1787826327.775408:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23781 00000100:00100000:3.0:1787826327.775416:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:24703:x1874663346244096:12345-0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826327.775432:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826327.775451:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b1af lrc: 2/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b1a8 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.775475:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b1af lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b1a8 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.775490:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b1af lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1a8 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.775498:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f79785cc700) 00010000:00010000:3.0:1787826327.775501:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826327.775504:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826327.775546:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346244096:12345-0@lo:101:mdt00_007.0 Request processed in 120us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.775552:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23781 00000100:00100000:3.0:1787826327.775684:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244096:0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826327.775699:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1af expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.775710:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1af expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826327.775724:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x194:0x0] 00000100:00100000:3.0:1787826327.775761:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244224:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826327.775790:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.775814:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58bcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.775830:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346244224 00000100:00100000:3.0:1787826327.775834:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346244224 00000100:00100000:3.0:1787826327.775858:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23782 00000100:00100000:3.0:1787826327.775865:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346244224:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826327.775930:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346244224:12345-0@lo:1000:mdt00_007.0 Request processed in 65us (139us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.775938:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23782 00000100:00100000:3.0:1787826327.775964:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244224:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826327.776011:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244352:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826327.776034:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.776049:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58bcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.776059:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346244352 00000100:00100000:3.0:1787826327.776062:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346244352 00000100:00100000:3.0:1787826327.776082:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23783 00000100:00100000:3.0:1787826327.776089:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346244352:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826327.776172:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346244352:12345-0@lo:1000:mdt00_007.0 Request processed in 84us (137us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.776178:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23783 00000100:00100000:3.0:1787826327.776199:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244352:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826327.776234:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244480:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826327.776257:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.776269:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58bcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.776279:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346244480 00000100:00100000:3.0:1787826327.776282:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346244480 00000100:00100000:3.0:1787826327.776298:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23784 00000100:00100000:3.0:1787826327.776303:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346244480:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826327.776371:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346244480:12345-0@lo:1000:mdt00_007.0 Request processed in 67us (112us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826327.776377:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23784 00000100:00100000:3.0:1787826327.776401:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244480:0@lo:1000:mdt00_007.0 00010000:00010000:3.0:1787826327.776447:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776455:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776464:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.776475:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244608:0@lo:101:mdt00_007.0 00000100:00100000:3.0:1787826327.776499:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.776513:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.776523:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346244608 00000100:00100000:3.0:1787826327.776526:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346244608 00000100:00100000:3.0:1787826327.776560:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23785 00000100:00100000:3.0:1787826327.776565:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346244608:12345-0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826327.776573:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826327.776586:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b1bd lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b1b6 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776603:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b1bd lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b1b6 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776614:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b1bd lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1b6 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776622:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826327.776624:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826327.776628:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826327.776656:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:24703:x1874663346244608:12345-0@lo:101:mdt00_007.0 Request processed in 91us (156us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.776662:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23785 00000100:00100000:3.0:1787826327.776686:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346244608:0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826327.776697:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776705:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776728:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776736:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.776743:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787826327.777056:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].223, 64 off22464 00000004:00080000:3.0:1787826327.777068:0:24703:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8139988961842ns, next at 8140988959035ns, rc = 0 00000040:00080000:3.0:1787826327.777094:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1208, 384 off1100152 00000040:00080000:3.0:1787826327.777166:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1527, 464 off1543352 00000004:00080000:3.0:1787826327.777171:0:24703:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784f58b840 with version 6845 00000004:00080000:3.0:1787826327.777204:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6845 rpc_version 6845 00000020:00100000:3.0:1787826327.777223:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826327.777232:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b660000 x1874663346244864/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826327.777248:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346244864:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826327.777287:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.777305:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.777319:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346244864 00000100:00100000:3.0:1787826327.777323:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346244864 00000100:00100000:3.0:1787826327.777350:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23786 00000100:00100000:3.0:1787826327.777357:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346244864:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826327.777602:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346244864:12345-0@lo:1000:osp_up0-1.0 Request processed in 244us (314us total) trans 8589957414 rc 0/0 00000100:00100000:3.0:1787826327.777609:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23786 00000100:00100000:3.0:1787826327.777653:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346244864:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787826327.777681:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777692:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777702:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777709:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777715:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777721:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777738:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826327.777759:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777766:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777773:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777779:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826327.777784:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777791:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777801:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:3.0:1787826327.777804:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826327.777807:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777813:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b1c4 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777824:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777832:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777839:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777859:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787826327.777868:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x194:0x0] 00010000:00010000:3.0:1787826327.777874:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777882:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777889:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777903:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787826327.777905:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x194:0x0] 00010000:00010000:3.0:1787826327.777910:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1af expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.777916:0:24703:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1af expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.777976:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225373:x1874663332618240:12345-192.168.203.56@tcp:36:rm.0 Request processed in 14818us (15012us total) trans 4294985421 rc 0/0 00000100:00100000:3.0:1787826327.777984:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68493 00010000:00010000:3.0:1787826327.778006:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826327.778010:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778017:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826327.778020:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778027:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778035:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:3.0:1787826327.778038:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787826327.778041:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778047:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b1cb lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e51:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778061:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826327.778065:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778071:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826327.778076:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826327.778084:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778124:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778131:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826327.778150:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778163:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4b1b6 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1bd expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778174:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826327.778177:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778183:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826327.778187:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826327.778194:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778209:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778214:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826327.778227:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 1/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778234:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efa00/0xe21309aa81f4b19a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1a1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778245:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826327.778246:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826327.778248:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826327.778253:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826327.778260:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826327.778270:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346244992:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.778305:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.778334:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346245120:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.778357:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.778389:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346244992 00000100:00100000:3.0:1787826327.778394:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346244992 00000100:00100000:3.0:1787826327.778423:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346245120 00000100:00100000:3.0:1787826327.778425:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346245120 00000100:00100000:3.0:1787826327.778442:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11122 00000100:00100000:3.0:1787826327.778450:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346244992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.778459:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826327.778475:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b1bd lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e51:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b1b6 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.778486:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e51:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826327.778492:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826327.778518:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346244992:12345-0@lo:103:ldlm_bl.0 Request processed in 70us (213us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.778524:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11122 00010000:00010000:1.0:1787826327.779771:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826327.779812:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11123 00000100:00100000:1.0:1787826327.779829:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346245120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826327.779846:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826327.779868:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b1a1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x194:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b19a expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.779884:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:1.0:1787826327.779894:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826327.779962:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346245120:12345-0@lo:103:ldlm_bl.0 Request processed in 140us (1604us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826327.779968:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11123 00000100:00100000:1.0:1787826327.780016:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346245120:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.781717:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b661180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346244992:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.785706:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826327.785967:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332618496 00000100:00100000:1.0:1787826327.785978:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332618496 00000100:00100000:1.0:1787826327.786052:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11124 00000100:00100000:1.0:1787826327.786063:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332618496:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:1.0:1787826327.786079:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826327.786127:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4aff6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015ec expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.786146:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f794385a700) 00010000:00010000:1.0:1787826327.786151:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826327.786207:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332618496:12345-192.168.203.56@tcp:103:rm.0 Request processed in 145us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826327.786215:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11124 00000100:00100000:3.0:1787826327.786423:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.786449:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332618752 00000100:00100000:3.0:1787826327.786453:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332618752 00000100:00100000:3.0:1787826327.786483:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9358 00000100:00100000:3.0:1787826327.786491:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225373:x1874663332618752:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:3.0:1787826327.786707:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:225373:x1874663332618752:12345-192.168.203.56@tcp:35:rm.0 Request processed in 216us (283us total) trans 4294985422 rc 0/0 00000100:00100000:3.0:1787826327.786714:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9358 00000100:00100000:0.0:1787826327.791169:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826327.791267:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332619008 00000100:00100000:1.0:1787826327.791277:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332619008 00000100:00100000:1.0:1787826327.791343:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68494 00000100:00100000:1.0:1787826327.791354:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:225373:x1874663332619008:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826327.791433:0:35095:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826327.791465:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b162 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e50:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801663 expref: 14 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.791485:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b12a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801647 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.791499:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b004 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015f3 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.791509:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:1.0:1787826327.791514:0:35095:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826327.791571:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.791580:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.791589:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826327.791615:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346245248:0@lo:101:mdt00_012.0 00000100:00100000:1.0:1787826327.791672:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826327.791719:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.791778:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346245248 00000100:00100000:0.0:1787826327.791784:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346245248 00000100:00100000:0.0:1787826327.791835:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23787 00000100:00100000:0.0:1787826327.791844:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346245248:12345-0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826327.791857:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826327.791871:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b1d2 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.791892:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.791913:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.791922:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.792142:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.792157:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 7 pid: 6529 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826327.792170:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346245376:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826327.792249:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.792259:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 7 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826327.792268:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346245504:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826327.792291:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453cec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826327.798858:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826327.799184:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332619136 00000100:00100000:2.0:1787826327.799194:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332619136 00010000:00010000:2.0:1787826327.799237:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 8 pid: 6529 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.799269:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21975 00000100:00100000:2.0:1787826327.799279:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332619136:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826327.799293:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826327.799301:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 8 pid: 6529 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.799315:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 8 pid: 6529 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.799340:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826327.799347:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.799357:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826327.799415:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332619136:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 138us (565us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.799423:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21975 00010000:00010000:2.0:1787826327.800218:0:6532:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b043 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080160f expref: 7 pid: 6529 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.800234:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346245376:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826327.800246:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346245504:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826327.800280:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.800289:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826327.800292:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826327.800296:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.800304:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:2.0:1787826327.800372:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35095:x1874663346245248:12345-0@lo:101:mdt00_012.0 Request processed in 8529us (8699us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.800380:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23787 00000100:00100000:3.0:1787826327.800911:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346245248:0@lo:101:mdt00_012.0 00010000:00010000:3.0:1787826327.800930:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.800951:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826327.802760:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.802886:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332619264 00000100:00100000:3.0:1787826327.802894:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332619264 00010000:00010000:3.0:1787826327.802928:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 8 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.803130:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21976 00000100:00100000:3.0:1787826327.803141:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332619264:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.803165:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826327.803173:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 8 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.803185:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 8 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.803209:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4afda lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608015e5 expref: 8 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.803230:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826327.803237:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.803256:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.803265:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.803300:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.803315:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346245632:0@lo:105: 00000100:00100000:3.0:1787826327.803362:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.803411:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826327.803445:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346245632 00000100:00100000:1.0:1787826327.803449:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346245632 00000100:00100000:1.0:1787826327.803489:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7878 00000100:00100000:1.0:1787826327.803497:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346245632:12345-0@lo:105: 00010000:00010000:1.0:1787826327.803508:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.803518:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.803525:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.803544:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.803555:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.803564:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.803622:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece600) 00010000:00010000:2.0:1787826327.803699:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.803708:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826327.803733:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826327.803771:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.803778:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.803791:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.803816:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc30380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346245760:0@lo:101:mdt00_012.0 00000100:00100000:2.0:1787826327.803851:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826327.803904:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826327.804177:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346245632:12345-0@lo:105: Request processed in 677us (810us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.804183:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346245632:0@lo:105: 00000100:00100000:1.0:1787826327.804185:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7878 00010000:00010000:3.0:1787826327.804197:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826327.804243:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346245760 00000100:00100000:3.0:1787826327.804243:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332619264:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1103us (1490us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826327.804248:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346245760 00000100:00100000:3.0:1787826327.804574:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21976 00000100:00100000:0.0:1787826327.804600:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23788 00000100:00100000:0.0:1787826327.804610:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:35095:x1874663346245760:12345-0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826327.804625:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826327.804641:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b1e7 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b1e0 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.804661:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b1e7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b1e0 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.804677:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b1e7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1e0 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.804684:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57af00) 00010000:00010000:0.0:1787826327.804686:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826327.804689:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826327.804732:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346245760:12345-0@lo:101:mdt00_012.0 Request processed in 125us (881us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826327.804737:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23788 00000100:00100000:3.0:1787826327.804797:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc30380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346245760:0@lo:101:mdt00_012.0 00010000:00010000:3.0:1787826327.804811:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.804825:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826327.804843:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826327.804877:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346245888:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826327.804913:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.804937:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.805023:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346245888 00000100:00100000:0.0:1787826327.805029:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346245888 00000100:00100000:0.0:1787826327.805057:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23789 00000100:00100000:0.0:1787826327.805070:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8bc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346245888:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826327.805148:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8bc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346245888:12345-0@lo:1000:mdt00_012.0 Request processed in 78us (234us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826327.805155:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23789 00000100:00100000:3.0:1787826327.805902:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346245888:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826327.805957:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346246016:0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826327.805992:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.806018:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826327.806045:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346246016 00000100:00100000:2.0:1787826327.806059:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346246016 00000100:00100000:2.0:1787826327.806099:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23790 00000100:00100000:2.0:1787826327.806129:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346246016:12345-0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826327.806239:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346246016:12345-0@lo:1000:mdt00_012.0 Request processed in 112us (246us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826327.806246:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23790 00000100:00100000:3.0:1787826327.806542:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346246016:0@lo:1000:mdt00_012.0 00010000:00010000:3.0:1787826327.806594:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.806602:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.806610:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.806619:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346246144:0@lo:101:mdt00_012.0 00000100:00100000:3.0:1787826327.806641:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.806659:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826327.806674:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346246144 00000100:00100000:2.0:1787826327.806679:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346246144 00000100:00100000:2.0:1787826327.806711:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23791 00000100:00100000:2.0:1787826327.806718:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:35095:x1874663346246144:12345-0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826327.806727:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826327.806743:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b1ee expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.806764:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.806772:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b1ee expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.806799:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.806880:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 6 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.806891:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346246272:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826327.807026:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826327.809849:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346246272:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826327.809893:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1ee expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.809906:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f794385a700) 00010000:00010000:2.0:1787826327.809910:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826327.809914:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1ee expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.809923:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826327.809985:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:35095:x1874663346246144:12345-0@lo:101:mdt00_012.0 Request processed in 3266us (3340us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.809993:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23791 00000100:00100000:3.0:1787826327.810037:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346246144:0@lo:101:mdt00_012.0 00010000:00010000:3.0:1787826327.810066:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.810077:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826327.811924:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.812725:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332619392 00000100:00100000:3.0:1787826327.812737:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332619392 00010000:00010000:3.0:1787826327.812782:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 7 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.812880:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21977 00000100:00100000:3.0:1787826327.812892:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332619392:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.812907:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826327.812915:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 7 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.812928:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 7 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813028:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b05f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801616 expref: 7 pid: 35093 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813043:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f794385a700) 00010000:00010000:3.0:1787826327.813049:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1ee expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813224:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b1ee expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813239:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b1ee expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813277:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1ee expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.813294:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346246400:0@lo:105: 00000100:00100000:3.0:1787826327.813358:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.813445:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826327.813482:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346246400 00000100:00100000:1.0:1787826327.813487:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346246400 00000100:00100000:1.0:1787826327.813573:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7879 00000100:00100000:1.0:1787826327.813583:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346246400:12345-0@lo:105: 00010000:00010000:1.0:1787826327.813595:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.813606:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.813615:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.813623:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.813636:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.813645:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.813673:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69600) 00000100:00100000:1.0:1787826327.813733:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346246400:12345-0@lo:105: Request processed in 153us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826327.813741:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7879 00010000:00010000:3.0:1787826327.813788:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813797:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.813803:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346246400:0@lo:105: 00010000:00010000:2.0:1787826327.813821:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826327.813838:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813848:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813857:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.813863:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332619392:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 975us (1947us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.813873:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21977 00010000:00010000:3.0:1787826327.813874:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813882:0:35095:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813890:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813913:0:35095:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.813924:0:35095:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 11 pid: 24703 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.813945:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001_UUID:35095:1874663346246528:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826327.813977:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.816929:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001_UUID:35095:1874663346246528:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826327.817198:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826327.818690:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826327.818741:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332619520 00000100:00100000:2.0:1787826327.818750:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332619520 00010000:00010000:2.0:1787826327.818793:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 12 pid: 24703 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.818841:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21978 00000100:00100000:2.0:1787826327.818851:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f6850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332619520:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826327.818866:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826327.818873:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 12 pid: 24703 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.818897:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 12 pid: 24703 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.818921:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b138 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080164e expref: 12 pid: 24703 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.818939:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f784abfe700) 00010000:00010000:2.0:1787826327.818943:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.818953:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.818959:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.819000:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826327.819065:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f6850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332619520:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 215us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.819072:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21978 00010000:00010000:0.0:1787826327.819093:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.819101:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.819165:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.819173:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.819188:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826327.819209:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346246656:0@lo:101:mdt00_012.0 00000100:00100000:0.0:1787826327.819250:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826327.819283:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453cef00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826327.819793:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346246656 00000100:00100000:2.0:1787826327.819809:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346246656 00000100:00100000:2.0:1787826327.819851:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23792 00000100:00100000:2.0:1787826327.819859:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:35095:x1874663346246656:12345-0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826327.819874:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826327.819900:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b211 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x194:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b20a expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.819922:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b211 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b20a expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.819934:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b211 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b20a expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.819942:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f784abfe800) 00010000:00010000:2.0:1787826327.819945:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826327.819948:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:2.0:1787826327.819994:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:35095:x1874663346246656:12345-0@lo:101:mdt00_012.0 Request processed in 135us (743us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.820000:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23792 00000100:00100000:0.0:1787826327.820022:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346246656:0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826327.820039:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.820050:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826327.820824:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346246784:0@lo:403:mdt00_012.0 00000100:00100000:0.0:1787826327.820873:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826327.820896:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453cef00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.820965:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346246784 00000100:00100000:3.0:1787826327.820971:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346246784 00000100:00100000:3.0:1787826327.821011:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23793 00000100:00100000:3.0:1787826327.821020:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35095:x1874663346246784:12345-0@lo:403:mdt00_012.0 00000100:00100000:3.0:1787826327.823503:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35095:x1874663346246784:12345-0@lo:403:mdt00_012.0 Request processed in 2474us (2620us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.823516:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23793 00000100:00100000:0.0:1787826327.823611:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346246784:0@lo:403:mdt00_012.0 00000040:00080000:0.0:1787826327.824810:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1209, 952 off1101104 00000040:00080000:0.0:1787826327.825089:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1528, 1032 off1544384 00000004:00080000:0.0:1787826327.825096:0:35095:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78453cef00 with version 6846 00000004:00080000:3.0:1787826327.825258:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6846 rpc_version 6846 00000020:00100000:3.0:1787826327.825293:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826327.825303:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f797c320380 x1874663346247040/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826327.825333:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346247040:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826327.825541:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.825602:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.825649:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346247040 00000100:00100000:0.0:1787826327.825656:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346247040 00000100:00100000:0.0:1787826327.825747:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23794 00000100:00100000:0.0:1787826327.825756:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346247040:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826327.826247:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346247040:12345-0@lo:1000:osp_up0-1.0 Request processed in 491us (713us total) trans 8589957415 rc 0/0 00000100:00100000:0.0:1787826327.826259:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23794 00000100:00100000:3.0:1787826327.826366:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346247040:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787826327.826511:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e50:0x0]: returning size 4096 00010000:00010000:1.0:1787826327.826544:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826555:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826562:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826569:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826327.826574:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826583:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826599:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x196:0x0].0x0 (ffff9f784abfe700) 00010000:00010000:1.0:1787826327.826604:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826327.826609:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826615:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b203 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826636:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826646:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826654:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826682:0:35095:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826327.826692:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x194:0x0] 00010000:00010000:1.0:1787826327.826700:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826706:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826712:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826717:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826327.826720:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826726:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826734:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:1.0:1787826327.826737:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826327.826740:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826745:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b1fc lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826755:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826763:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826769:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826782:0:35095:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787826327.826792:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826327.826796:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826803:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826810:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826821:0:35095:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787826327.826826:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826327.826830:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.826837:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826327.826915:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225373:x1874663332619008:12345-192.168.203.56@tcp:36:rm.0 Request processed in 35560us (35752us total) trans 4294985423 rc 0/0 00000100:00100000:1.0:1787826327.826923:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68494 00010000:00010000:3.0:1787826327.827125:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826327.827133:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827141:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826327.827146:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826327.827154:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827184:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 2/0,0 mode: --/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827192:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826327.827219:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 1/0,0 mode: --/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827227:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4b20a lrc: 0/0,0 mode: --/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b211 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827240:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826327.827243:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827249:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826327.827253:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826327.827260:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827273:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827279:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826327.827296:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827302:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b1ee lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1f5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827313:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826327.827316:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827321:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826327.827323:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826327.827329:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827346:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827352:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826327.827356:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827362:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece600/0xe21309aa81f4b1d2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1d9 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827369:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826327.827370:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826327.827372:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826327.827380:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826327.827393:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346247168:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.827437:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.827477:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346247424:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.827502:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.827521:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826327.827526:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346247296:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.827560:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.827591:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346247168 00000100:00100000:3.0:1787826327.827597:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346247168 00000100:00100000:3.0:1787826327.827639:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346247424 00000100:00100000:3.0:1787826327.827642:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346247424 00000100:00100000:3.0:1787826327.827660:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346247296 00000100:00100000:3.0:1787826327.827663:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346247296 00000100:00100000:3.0:1787826327.827682:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11125 00000100:00100000:3.0:1787826327.827690:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+17:3654:x1874663346247168:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.827702:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826327.827719:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b211 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x194:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b20a expref: 17 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827731:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x0 (ffff9f784abfe800) 00010000:00010000:3.0:1787826327.827740:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826327.827771:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3654:x1874663346247168:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.827777:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11125 00000100:00100000:3.0:1787826327.827794:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11126 00000100:00100000:3.0:1787826327.827800:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3653:x1874663346247424:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.827807:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826327.827819:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b1d9 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b1d2 expref: 15 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827830:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826327.827832:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826327.827858:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+14:3653:x1874663346247424:12345-0@lo:103:ldlm_bl.0 Request processed in 59us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.827864:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11126 00000100:00100000:3.0:1787826327.827876:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11127 00000100:00100000:3.0:1787826327.827882:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346247296:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.827889:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826327.827900:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b1f5 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e50:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b1ee expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.827914:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f794385a700) 00010000:00010000:3.0:1787826327.827920:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826327.827942:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346247296:12345-0@lo:103:ldlm_bl.0 Request processed in 61us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.827948:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11127 00010000:00010000:3.0:1787826327.827964:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826327.827986:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346247168:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.828012:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346247296:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826327.828019:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346247424:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.828037:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826327.920922:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826327.921022:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332619648 00000100:00100000:2.0:1787826327.921032:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332619648 00000100:00100000:2.0:1787826327.921150:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68495 00000100:00100000:2.0:1787826327.921162:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225381:x1874663332619648:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826327.921221:0:35098:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345561600 00000100:00100000:2.0:1787826327.921529:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225381:x1874663332619648:12345-192.168.203.56@tcp:41:stat.0 Request processed in 368us (617us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.921551:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68495 00000100:00100000:1.0:1787826327.963432:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826327.963552:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332619776 00000100:00100000:0.0:1787826327.963562:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332619776 00000100:00100000:0.0:1787826327.963630:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68496 00000100:00100000:0.0:1787826327.963641:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225385:x1874663332619776:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826327.963662:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826327.963684:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b218 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801678 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.963761:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.963775:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.963790:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826327.963843:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826327.963847:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826327.963869:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.963896:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b218 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.963915:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.963929:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826327.963933:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826327.963936:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826327.964008:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225385:x1874663332619776:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 369us (585us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826327.964015:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68496 00000100:00100000:1.0:1787826327.966265:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.966355:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332619904 00000100:00100000:3.0:1787826327.966366:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332619904 00000100:00100000:3.0:1787826327.966428:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68497 00000100:00100000:3.0:1787826327.966438:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225385:x1874663332619904:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826327.966458:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826327.966477:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b226 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080167f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.966558:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.966572:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.966583:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.966599:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.966609:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b1e7 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b1e0 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.966618:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.966654:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b1e7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b1e0 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.966671:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6528:1874663346247552:0@lo:104: 00000100:00100000:3.0:1787826327.966725:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826327.966770:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826327.966872:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346247552 00000100:00100000:1.0:1787826327.966877:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346247552 00000100:00100000:1.0:1787826327.966916:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7880 00000100:00100000:1.0:1787826327.966924:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6528:x1874663346247552:12345-0@lo:104: 00010000:00010000:1.0:1787826327.966938:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826327.967011:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826327.967018:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6528:x1874663346247552:12345-0@lo:104: Request processed in 95us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826327.967025:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7880 00010000:00010000:2.0:1787826327.967092:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826327.967100:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.967143:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826327.967149:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826327.967157:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.967183:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.967189:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826327.967208:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.967215:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b1e0 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b1e7 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.967234:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826327.967236:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826327.967238:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826327.967314:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826327.967327:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346247680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826327.967360:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826327.967445:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346247680 00000100:00100000:2.0:1787826327.967449:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346247680 00010000:00010000:2.0:1787826327.967469:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b1e7 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b1e0 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.967499:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21979 00000100:00100000:2.0:1787826327.967507:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346247680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826327.967517:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826327.967922:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6528:1874663346247552:0@lo:104: 00010000:00010000:3.0:1787826327.967943:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826327.969979:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957415 is committed 00010000:00010000:2.0:1787826327.970254:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b1e7 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b1e0 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.970276:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57af00) 00010000:00010000:2.0:1787826327.970282:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.970293:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.970301:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.970383:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826327.970475:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346247680:12345-0@lo:103:ldlm_bl.0 Request processed in 2967us (3112us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826327.970486:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21979 00000100:00100000:1.0:1787826327.970594:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346247680:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826327.970649:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970657:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970722:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970729:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970736:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970742:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970748:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826327.970753:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970761:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970771:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826327.970774:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826327.970777:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970784:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b22d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970794:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970903:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970911:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970916:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826327.970920:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970927:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970935:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57af00) 00010000:00010000:3.0:1787826327.970938:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826327.970940:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970946:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b234 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.970991:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b226 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080167f expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.971001:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826327.971003:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826327.971007:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b226 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080167f expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.971017:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826327.971242:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225385:x1874663332619904:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 4804us (4985us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826327.971251:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68497 00000100:00100000:1.0:1787826327.975743:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826327.975836:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332620032 00000100:00100000:3.0:1787826327.975846:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332620032 00000100:00100000:3.0:1787826327.975905:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68498 00000100:00100000:3.0:1787826327.975917:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225385:x1874663332620032:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826327.975937:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826327.975959:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b23b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801686 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.976032:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.976046:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.976066:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826327.976164:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826327.976168:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826327.976190:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.976217:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b23b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.976235:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.976243:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826327.976247:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826327.976250:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826327.976317:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225385:x1874663332620032:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 403us (584us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826327.976324:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68498 00000100:00100000:1.0:1787826327.989432:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826327.989530:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332620288 00000100:00100000:2.0:1787826327.989550:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332620288 00000100:00100000:2.0:1787826327.989616:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68499 00000100:00100000:2.0:1787826327.989627:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225385:x1874663332620288:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:2.0:1787826327.989750:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346247808:0@lo:1000:mdt00_009.0 00000100:00100000:2.0:1787826327.989807:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826327.989858:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849780000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.989989:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346247808 00000100:00100000:0.0:1787826327.989995:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346247808 00000100:00100000:0.0:1787826327.990035:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23795 00000100:00100000:0.0:1787826327.990044:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346247808:12345-0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826327.990157:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346247808:12345-0@lo:1000:mdt00_009.0 Request processed in 114us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826327.990165:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23795 00000100:00100000:1.0:1787826327.990243:0:35092:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957415 gen 2 00000004:00080000:1.0:1787826327.990248:0:35092:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957413 committed transno 8589957415 00000004:00080000:1.0:1787826327.990269:0:35092:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957414 committed transno 8589957415 00000004:00080000:1.0:1787826327.990280:0:35092:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957415 committed transno 8589957415 00000100:00100000:1.0:1787826327.990294:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346247808:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826327.990335:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346247936:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826327.990377:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826327.990403:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.990492:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346247936 00000100:00100000:0.0:1787826327.990496:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346247936 00000100:00100000:0.0:1787826327.990863:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23796 00000100:00100000:0.0:1787826327.990872:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346247936:12345-0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826327.990962:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346247936:12345-0@lo:1000:mdt00_009.0 Request processed in 92us (588us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826327.990969:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23796 00000100:00100000:2.0:1787826327.992692:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346247936:0@lo:1000:mdt00_009.0 00010000:00010000:2.0:1787826327.992777:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.992787:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826327.992799:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826327.992819:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33b80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248064:0@lo:101:mdt00_009.0 00000100:00100000:2.0:1787826327.992883:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826327.992921:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844274540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.993776:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346248064 00000100:00100000:0.0:1787826327.993786:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346248064 00000100:00100000:0.0:1787826327.993953:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23797 00000100:00100000:0.0:1787826327.993964:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346248064:12345-0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826327.993987:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826327.994020:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b249 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.994051:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.994060:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.994068:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.994096:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.994137:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 7 pid: 35094 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826327.994150:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346248192:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826327.994240:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826327.994249:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 6 pid: 8752 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826327.994258:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346248320:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826327.994284:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.999163:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346248320:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826328.000509:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826328.000895:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346248192:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826328.000938:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.000953:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826328.000956:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826328.000960:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.000971:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826328.001157:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:35092:x1874663346248064:12345-0@lo:101:mdt00_009.0 Request processed in 7194us (8272us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826328.001166:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23797 00000100:00100000:3.0:1787826328.002625:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332620416 00000100:00100000:2.0:1787826328.003523:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33b80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248064:0@lo:101:mdt00_009.0 00010000:00010000:2.0:1787826328.003565:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.003579:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826328.005923:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332620416 00010000:00010000:3.0:1787826328.006021:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 7 pid: 8752 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826328.006073:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21980 00000100:00100000:3.0:1787826328.006088:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332620416:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826328.006130:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826328.006147:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 7 pid: 8752 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.006164:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 7 pid: 8752 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826328.006179:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:3.0:1787826328.006190:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b21f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801678 expref: 7 pid: 8752 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.006207:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826328.006215:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826328.006265:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787826328.006349:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826328.006413:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332620416:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 330us (5910us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.006421:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21980 00000100:00100000:3.0:1787826328.006441:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332620544 00000100:00100000:3.0:1787826328.006456:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332620544 00010000:00010000:3.0:1787826328.006483:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 8 pid: 35094 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826328.006522:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826328.006820:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21981 00000100:00100000:2.0:1787826328.007218:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332620544:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826328.007230:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826328.007238:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 8 pid: 35094 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.007251:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 8 pid: 35094 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.007269:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b242 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801686 expref: 8 pid: 35094 timeout: 8240 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.007282:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826328.007287:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.007296:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.007305:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.007340:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826328.007424:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346248448:0@lo:105: 00000100:00100000:2.0:1787826328.007479:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826328.007588:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844274540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826328.007637:0:6509:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826328.007651:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346248448 00000100:00100000:3.0:1787826328.007658:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346248448 00000100:00100000:3.0:1787826328.007743:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7881 00000100:00100000:3.0:1787826328.007752:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78557e8850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346248448:12345-0@lo:105: 00010000:00010000:3.0:1787826328.007763:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.007774:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.007782:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.007790:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.007797:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.007805:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.007874:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eee00) 00000100:00100000:3.0:1787826328.007924:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78557e8850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346248448:12345-0@lo:105: Request processed in 174us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.007935:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7881 00000100:00100000:0.0:1787826328.008044:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346248448:0@lo:105: 00010000:00010000:0.0:1787826328.008062:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826328.008102:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332620544:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 891us (1925us total) trans 0 rc 0/0 00010000:00010000:2.0:1787826328.008464:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.008475:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826328.008498:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826328.008542:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.008549:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.008558:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826328.008576:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33b80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248576:0@lo:101:mdt00_009.0 00000100:00100000:2.0:1787826328.008614:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.008619:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21981 00000100:00100000:2.0:1787826328.008652:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844274540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826328.008717:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346248576 00000100:00100000:1.0:1787826328.008723:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346248576 00000100:00100000:1.0:1787826328.008770:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23798 00000100:00100000:1.0:1787826328.008780:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35092:x1874663346248576:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826328.008797:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826328.008813:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b25e lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b257 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.008982:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b25e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b257 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.008997:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b25e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b257 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.009057:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1900) 00010000:00010000:1.0:1787826328.009062:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826328.009065:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826328.009142:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346248576:12345-0@lo:101:mdt00_009.0 Request processed in 363us (528us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826328.009148:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23798 00000100:00100000:0.0:1787826328.013530:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33b80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248576:0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826328.013575:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.013597:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826328.013625:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826328.013703:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248704:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.013780:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.013813:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453d9900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826328.013995:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346248704 00000100:00100000:1.0:1787826328.014013:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346248704 00000100:00100000:1.0:1787826328.014073:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23799 00000100:00100000:1.0:1787826328.014083:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346248704:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826328.014442:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346248704:12345-0@lo:1000:mdt00_009.0 Request processed in 359us (660us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826328.014450:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23799 00000100:00100000:0.0:1787826328.014528:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248704:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.014593:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248832:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.014622:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.014654:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826328.014748:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346248832 00000100:00100000:1.0:1787826328.014753:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346248832 00000100:00100000:1.0:1787826328.014791:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23800 00000100:00100000:1.0:1787826328.014800:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346248832:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826328.014900:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346248832:12345-0@lo:1000:mdt00_009.0 Request processed in 101us (277us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826328.014907:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23800 00000100:00100000:0.0:1787826328.015011:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248832:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.015090:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248960:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.015338:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.015364:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500b1840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826328.015410:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346248960 00000100:00100000:2.0:1787826328.015415:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346248960 00000100:00100000:2.0:1787826328.015452:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23801 00000100:00100000:2.0:1787826328.015461:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346248960:12345-0@lo:1000:mdt00_009.0 00000100:00100000:2.0:1787826328.015547:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346248960:12345-0@lo:1000:mdt00_009.0 Request processed in 87us (209us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826328.015553:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23801 00000100:00100000:0.0:1787826328.015988:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346248960:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.016061:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346249088:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.016092:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.016421:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453d9900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826328.018727:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346249088 00000100:00100000:2.0:1787826328.018742:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346249088 00000100:00100000:2.0:1787826328.018880:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23802 00000100:00100000:2.0:1787826328.018890:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346249088:12345-0@lo:1000:mdt00_009.0 00000100:00100000:2.0:1787826328.019151:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346249088:12345-0@lo:1000:mdt00_009.0 Request processed in 262us (3057us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826328.019159:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23802 00000100:00100000:0.0:1787826328.019373:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346249088:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.019451:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346249216:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.019488:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.019589:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826328.019624:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346249216 00000100:00100000:3.0:1787826328.019629:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346249216 00000100:00100000:3.0:1787826328.019666:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23803 00000100:00100000:3.0:1787826328.019673:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346249216:12345-0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826328.019733:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346249216:12345-0@lo:1000:mdt00_009.0 Request processed in 61us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.019739:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23803 00000100:00100000:0.0:1787826328.019972:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346249216:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.020087:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843091180 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346249344:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826328.020350:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.020418:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500b1840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826328.020454:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346249344 00000100:00100000:3.0:1787826328.020459:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346249344 00000100:00100000:3.0:1787826328.020542:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23804 00000100:00100000:3.0:1787826328.020551:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346249344:12345-0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826328.020614:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346249344:12345-0@lo:1000:mdt00_009.0 Request processed in 65us (264us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.020621:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23804 00000100:00100000:0.0:1787826328.020686:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843091180 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346249344:0@lo:1000:mdt00_009.0 00000040:00080000:0.0:1787826328.021721:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1210, 2168 off1103272 00000040:00080000:0.0:1787826328.021812:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1529, 2368 off1546752 00000004:00080000:0.0:1787826328.021820:0:35092:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78453d9900 with version 6847 00000004:00080000:1.0:1787826328.022233:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6847 rpc_version 6847 00000020:00100000:1.0:1787826328.022275:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826328.022284:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f79785e6680 x1874663346249600/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826328.022308:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346249600:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826328.022378:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826328.024788:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826328.024845:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346249600 00000100:00100000:2.0:1787826328.024856:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346249600 00000100:00100000:3.0:1787826328.024937:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23805 00000100:00100000:3.0:1787826328.024950:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346249600:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826328.026041:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346249600:12345-0@lo:1000:osp_up0-1.0 Request processed in 1092us (3658us total) trans 8589957416 rc 0/0 00000100:00100000:3.0:1787826328.026050:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23805 00000100:00100000:1.0:1787826328.028239:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346249600:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826328.029505:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.029525:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.029548:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.029572:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.029579:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.029585:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.029623:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.029632:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.029744:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826328.029769:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843091180 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346249728:0@lo:101:mdt00_009.0 00000100:00100000:0.0:1787826328.029841:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.029938:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826328.030102:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346249728 00000100:00100000:3.0:1787826328.030129:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346249728 00000100:00100000:3.0:1787826328.030185:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23806 00000100:00100000:3.0:1787826328.030194:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:35092:x1874663346249728:12345-0@lo:101:mdt00_009.0 00010000:00010000:3.0:1787826328.030213:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826328.030232:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b27a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b273 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.030250:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b27a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b273 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.030261:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b27a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b273 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.030269:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826328.030272:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826328.030274:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826328.030313:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35092:x1874663346249728:12345-0@lo:101:mdt00_009.0 Request processed in 121us (473us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.030319:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23806 00000100:00100000:0.0:1787826328.030477:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843091180 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346249728:0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826328.030492:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030501:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826328.030527:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001718f85c reply state 00000000b7ff3097 transno 4294985424 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030549:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030555:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826328.030558:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826328.030561:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826328.030567:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030574:0:35092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826328.030589:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x195:0x0] 00000004:00010000:0.0:1787826328.030597:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001718f85c reply state 00000000b7ff3097 transno 4294985424 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030604:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030610:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:0.0:1787826328.030612:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826328.030613:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787826328.030619:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e52:0x0]: returning size 4096 00000004:00080000:0.0:1787826328.030628:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826328.030636:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030655:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030662:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030683:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826328.030689:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826328.030693:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.030700:0:35092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826328.030783:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225385:x1874663332620288:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 41157us (41361us total) trans 4294985424 rc 0/0 00000100:00100000:0.0:1787826328.030791:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68499 00010000:00010000:3.0:1787826328.031193:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826328.031201:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.031210:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826328.031216:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826328.031224:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.031258:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.031269:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826328.031302:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.031309:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b249 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b250 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.031323:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826328.031325:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826328.031326:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826328.031334:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826328.031347:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346249856:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826328.031395:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826328.031432:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346249856 00000100:00100000:3.0:1787826328.031438:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346249856 00000100:00100000:3.0:1787826328.031484:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11128 00000100:00100000:3.0:1787826328.031492:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346249856:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826328.031501:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826328.031518:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b250 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b249 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.031541:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826328.031546:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826328.031575:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346249856:12345-0@lo:103:ldlm_bl.0 Request processed in 84us (181us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.031581:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11128 00000100:00100000:3.0:1787826328.032017:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346249856:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826328.038684:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826328.038891:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332620672 00000100:00100000:2.0:1787826328.038902:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332620672 00000100:00100000:2.0:1787826328.039070:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68500 00000100:00100000:2.0:1787826328.039081:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225374:x1874663332620672:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826328.039175:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826328.039199:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b281 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080168d expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.039267:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b288 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.039281:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b288 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.039302:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b288 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826328.039355:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826328.039359:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826328.039379:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b288 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.039406:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b281 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080168d expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.039423:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b288 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080168d expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.039431:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826328.039435:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826328.039438:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826328.039548:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225374:x1874663332620672:12345-192.168.203.56@tcp:101:bash.0 Request processed in 469us (874us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826328.039556:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68500 00000100:00100000:0.0:1787826328.049277:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826328.049420:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332620800 00000100:00100000:2.0:1787826328.049430:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332620800 00000100:00100000:2.0:1787826328.049494:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68501 00000100:00100000:2.0:1787826328.049503:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225374:x1874663332620800:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826328.049522:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826328.049553:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b28f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801694 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.049619:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.049633:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.049643:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.049653:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.049662:0:35093:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b25e lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b257 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.049670:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.049706:0:35093:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b25e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b257 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826328.049721:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000_UUID:35093:1874663346249984:0@lo:104: 00000100:00100000:2.0:1787826328.049773:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826328.049811:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844274540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826328.050018:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346249984 00000100:00100000:0.0:1787826328.050022:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346249984 00000100:00100000:0.0:1787826328.050059:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7882 00000100:00100000:0.0:1787826328.050067:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:35093:x1874663346249984:12345-0@lo:104: 00010000:00010000:0.0:1787826328.050080:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.050170:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826328.050176:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:35093:x1874663346249984:12345-0@lo:104: Request processed in 111us (403us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826328.050184:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7882 00010000:00010000:3.0:1787826328.050434:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826328.050442:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.050453:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826328.050459:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826328.050469:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.050497:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.050505:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826328.050552:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.050561:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef600/0xe21309aa81f4b257 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b25e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.050584:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826328.050586:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826328.050588:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826328.050767:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826328.050783:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346250112:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826328.050831:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826328.050969:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346250112 00000100:00100000:3.0:1787826328.050974:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346250112 00010000:00010000:3.0:1787826328.051000:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b25e lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b257 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826328.051040:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21982 00000100:00100000:3.0:1787826328.051048:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346250112:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826328.051059:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787826328.051503:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000_UUID:35093:1874663346249984:0@lo:104: 00010000:00010000:2.0:1787826328.051524:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826328.053601:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957416 is committed 00010000:00010000:3.0:1787826328.053904:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b25e lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b257 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.053929:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1900) 00010000:00010000:3.0:1787826328.053935:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.053946:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.053954:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054021:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826328.054200:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346250112:12345-0@lo:103:ldlm_bl.0 Request processed in 3150us (3373us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.054209:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21982 00010000:00010000:2.0:1787826328.054275:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.054284:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826328.054434:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-mdtlov_UUID:35093:1874663346250240:0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826328.054482:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826328.054517:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845dd10c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826328.054583:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346250240 00000100:00100000:3.0:1787826328.054589:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346250240 00000100:00100000:3.0:1787826328.054639:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23807 00000100:00100000:3.0:1787826328.054647:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:35093:x1874663346250240:12345-0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826328.054729:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:35093:x1874663346250240:12345-0@lo:1000:mdt00_010.0 Request processed in 83us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.054737:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23807 00000100:00100000:3.0:1787826328.054788:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-mdtlov_UUID:35093:1874663346250240:0@lo:1000:mdt00_010.0 00010000:00010000:3.0:1787826328.054883:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b2a4 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054896:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b2a4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054905:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b2a4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826328.054929:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b2a4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054943:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054950:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054956:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054961:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054967:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826328.054972:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054980:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.054990:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826328.054993:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826328.054997:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055003:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b296 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055016:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055022:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055027:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055032:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826328.055035:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055041:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055048:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1900) 00010000:00010000:3.0:1787826328.055051:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826328.055054:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055059:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b29d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055094:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b28f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801694 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055133:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b2a4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801694 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.055140:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:3.0:1787826328.055142:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826328.055144:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826328.055195:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225374:x1874663332620800:12345-192.168.203.56@tcp:101:bash.0 Request processed in 5692us (5930us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.055202:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68501 00000100:00100000:1.0:1787826328.055634:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346250112:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826328.058392:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826328.058559:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332620928 00000100:00100000:2.0:1787826328.058579:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332620928 00000100:00100000:2.0:1787826328.058645:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68502 00000100:00100000:2.0:1787826328.058656:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225374:x1874663332620928:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826328.058691:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826328.058715:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4b2ab lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080169b expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.058794:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b2b2 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.058817:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b2b2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.058986:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b2b2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826328.059127:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e52:0x0]: returning size 4096 00000004:00200000:2.0:1787826328.059133:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826328.059174:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4b2ab lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080169b expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.059194:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b2b2 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080169b expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.059203:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:2.0:1787826328.059206:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826328.059210:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826328.059269:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225374:x1874663332620928:12345-192.168.203.56@tcp:101:bash.0 Request processed in 615us (886us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826328.059275:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68502 00000100:00080000:0.0:1787826328.059996:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826328.064398:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826328.064553:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332621056 00000100:00100000:0.0:1787826328.064564:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332621056 00000100:00100000:0.0:1787826328.064632:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68503 00000100:00100000:0.0:1787826328.064643:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225374:x1874663332621056:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826328.064665:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826328.064687:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b2b9 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016a2 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.064763:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b2c0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.064779:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b2c0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.064791:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b2c0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826328.064861:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x197:0x0]: returning size 4096 00000004:00200000:0.0:1787826328.064866:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826328.064905:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b2b9 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016a2 expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.064924:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b2c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016a2 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.064933:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826328.064936:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826328.064940:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826328.065004:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225374:x1874663332621056:12345-192.168.203.56@tcp:101:bash.0 Request processed in 363us (615us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826328.065011:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68503 00000100:00100000:2.0:1787826328.069780:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826328.069931:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332621184 00000100:00100000:0.0:1787826328.069948:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332621184 00000100:00100000:0.0:1787826328.070008:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68504 00000100:00100000:0.0:1787826328.070020:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225374:x1874663332621184:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826328.070041:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826328.070064:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b2c7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016a9 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.070171:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 3/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.070186:0:11629:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b27a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b273 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.070196:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 4/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.070230:0:11629:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b27a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b273 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826328.070247:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346250368:0@lo:104: 00000100:00100000:0.0:1787826328.070435:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.070474:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826328.070734:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346250368 00000100:00100000:1.0:1787826328.070740:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346250368 00000100:00100000:1.0:1787826328.070792:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7883 00000100:00100000:1.0:1787826328.070801:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346250368:12345-0@lo:104: 00010000:00010000:1.0:1787826328.070813:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.071042:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826328.071050:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346250368:12345-0@lo:104: Request processed in 251us (616us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826328.071057:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7883 00000100:00100000:3.0:1787826328.071508:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346250368:0@lo:104: 00010000:00010000:3.0:1787826328.071542:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 3/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.075123:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826328.075139:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.075156:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826328.075164:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826328.075174:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.075231:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.075238:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826328.075327:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346250496:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826328.075404:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826328.075499:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346250496 00000100:00100000:0.0:1787826328.075506:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346250496 00010000:00010000:0.0:1787826328.075553:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b27a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b273 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826328.075597:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21983 00000100:00100000:0.0:1787826328.075607:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346250496:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826328.075619:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826328.076400:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b27a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b273 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.076418:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:0.0:1787826328.076424:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 3/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.076434:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.076441:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.076473:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826328.076563:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.076573:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826328.076630:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x195:0x0]: returning size 4096 00000004:00200000:3.0:1787826328.076635:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826328.076680:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b2c7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016a9 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.076698:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016a9 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.076711:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826328.076714:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826328.076717:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:3.0:1787826328.076780:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225374:x1874663332621184:12345-192.168.203.56@tcp:101:bash.0 Request processed in 6761us (7011us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.076789:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68504 00010000:00010000:2.0:1787826328.077601:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.077614:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43c00/0xe21309aa81f4b273 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b27a expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.077636:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826328.077638:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826328.077640:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826328.082582:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346250496:12345-0@lo:103:ldlm_bl.0 Request processed in 6965us (7166us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826328.082600:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21983 00000100:00100000:1.0:1787826328.085481:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346250496:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826328.093967:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826328.094491:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332621440 00000100:00100000:0.0:1787826328.094501:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332621440 00000100:00100000:0.0:1787826328.094575:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68505 00000100:00100000:0.0:1787826328.094586:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225374:x1874663332621440:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826328.094608:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826328.094611:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826328.094637:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b2ce lrc: 0/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016a9 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.094660:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:0.0:1787826328.094672:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826328.094687:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b2d5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016b0 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.094819:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 3/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.095647:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.095725:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.095760:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.095769:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.095776:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826328.096326:0:17813:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x521:0x0], index 1: 0 00010000:00010000:0.0:1787826328.096585:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b2ea lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096598:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b2ea lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096606:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b2ea lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096639:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096647:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096653:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096659:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096675:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826328.096680:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096686:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096697:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f784b4a5300) 00010000:00010000:0.0:1787826328.096699:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826328.096703:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 3/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096710:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b2dc lrc: 0/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826328.096723:0:17813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009883021e reply state 0000000080ab66c2 transno 8589957417 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096733:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096738:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826328.096740:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826328.096743:0:17813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826328.096785:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b2d5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016b0 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096811:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b2ea lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016b0 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826328.096819:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:0.0:1787826328.096821:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826328.096824:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826328.096911:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225374:x1874663332621440:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2325us (2952us total) trans 8589957417 rc 0/0 00000100:00100000:0.0:1787826328.096920:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68505 00000100:00100000:0.0:1787826328.102851:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826328.102922:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332621568 00000100:00100000:2.0:1787826328.102933:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332621568 00000100:00100000:2.0:1787826328.103071:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30844 00000100:00100000:2.0:1787826328.103083:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225374:x1874663332621568:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826328.103102:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826328.103218:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016b7 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826328.103432:0:35096:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x521:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00000100:00080000:0.0:1787826328.103448:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787826328.103452:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=8140s v=(0 0) 00010000:00010000:1.0:1787826328.103457:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826328.103465:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 9 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.103483:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 9 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.103493:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:1.0:1787826328.103561:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225374:x1874663332621568:12345-192.168.203.56@tcp:101:bash.0 Request processed in 481us (721us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826328.103568:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30844 00000100:00100000:2.0:1787826328.109363:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826328.109509:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332621824 00000100:00100000:3.0:1787826328.109519:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332621824 00000100:00100000:3.0:1787826328.109594:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9359 00000100:00100000:3.0:1787826328.109605:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225374:x1874663332621824:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826328.109927:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225374:x1874663332621824:12345-192.168.203.56@tcp:35:bash.0 Request processed in 322us (721us total) trans 8589957418 rc 0/0 00000100:00100000:3.0:1787826328.109935:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9359 00000100:00100000:0.0:1787826328.115551:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826328.115747:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332622080 00000100:00100000:3.0:1787826328.115757:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332622080 00000100:00100000:3.0:1787826328.115929:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68506 00000100:00100000:3.0:1787826328.115940:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225374:x1874663332622080:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826328.115960:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826328.115962:0:6529:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826328.116229:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b2c0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016a2 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.116249:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:3.0:1787826328.116254:0:6529:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826328.116270:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b2ff lrc: 2/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016be expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.116420:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.116432:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.116441:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.116456:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.116463:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.116469:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826328.116613:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346250624:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826328.116684:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826328.116722:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826328.117269:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346250624 00000100:00100000:1.0:1787826328.117277:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346250624 00000100:00100000:1.0:1787826328.117334:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23808 00000100:00100000:1.0:1787826328.117343:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346250624:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826328.117442:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346250624:12345-0@lo:1000:mdt00_002.0 Request processed in 100us (757us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826328.117450:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23808 00000100:00100000:3.0:1787826328.117795:0:6529:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957416 gen 2 00000004:00080000:3.0:1787826328.117812:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957416 committed transno 8589957416 00000100:00100000:3.0:1787826328.117846:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346250624:0@lo:1000:mdt00_002.0 00000004:00080000:3.0:1787826328.118167:0:6529:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4ce:0x0], index 0: 0 00010000:00010000:3.0:1787826328.118340:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b314 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118360:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b314 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118369:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b314 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118424:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118430:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118436:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118443:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118448:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826328.118453:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118463:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118476:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:3.0:1787826328.118479:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826328.118484:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118490:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b306 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826328.118507:0:6529:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008830c468 reply state 00000000c21b2a02 transno 4294985425 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118517:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118522:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x4ec94fd6 (ffff9f79785cce00) 00010000:00010000:3.0:1787826328.118524:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826328.118527:0:6529:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826328.118570:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4b2ff lrc: 0/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016be expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118584:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b314 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016be expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.118592:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f79785ccb00) 00010000:00010000:3.0:1787826328.118594:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826328.118596:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826328.118669:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225374:x1874663332622080:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2723us (3130us total) trans 4294985425 rc 0/0 00000100:00100000:3.0:1787826328.118678:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68506 00000100:00100000:0.0:1787826328.126230:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826328.127198:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332622208 00000100:00100000:2.0:1787826328.127212:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332622208 00000100:00100000:2.0:1787826328.127290:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30845 00000100:00100000:2.0:1787826328.127301:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:225374:x1874663332622208:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826328.127320:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826328.127344:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016c5 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826328.127373:0:24218:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4ce:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826328.127396:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ce:0x0].0x0 since=8140s v=(0 0) 00010000:00010000:2.0:1787826328.127401:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ce:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826328.127411:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.127429:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826328.127450:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00080000:0.0:1787826328.127483:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826328.127526:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225374:x1874663332622208:12345-192.168.203.56@tcp:101:bash.0 Request processed in 228us (1307us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826328.127544:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30845 00000100:00100000:2.0:1787826328.130995:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826328.131136:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332622464 00000100:00100000:1.0:1787826328.131147:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332622464 00000100:00100000:1.0:1787826328.131250:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9360 00000100:00100000:1.0:1787826328.131262:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225374:x1874663332622464:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:1.0:1787826328.131622:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225374:x1874663332622464:12345-192.168.203.56@tcp:35:bash.0 Request processed in 360us (634us total) trans 4294985426 rc 0/0 00000100:00100000:1.0:1787826328.131630:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9360 00000100:00100000:0.0:1787826328.160311:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826328.160656:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332622592 00000100:00100000:3.0:1787826328.160666:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332622592 00000100:00100000:3.0:1787826328.160752:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11129 00000100:00100000:3.0:1787826328.160764:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:6634:x1874663332622592:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826328.160783:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826328.160945:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b288 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080168d expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.160968:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826328.160981:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4b2ea lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016b0 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.160991:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f784b4a5600) 00010000:00010000:3.0:1787826328.161007:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b2a4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801694 expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.161015:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:3.0:1787826328.161022:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826328.161238:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:6634:x1874663332622592:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 476us (934us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.161246:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11129 00000100:00100000:0.0:1787826328.164416:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826328.164529:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332622848 00000100:00100000:1.0:1787826328.164548:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332622848 00000100:00100000:1.0:1787826328.164618:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11130 00000100:00100000:1.0:1787826328.164629:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:15673:x1874663332622848:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826328.164646:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826328.164672:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b314 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016be expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.164692:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f79785ccb00) 00010000:00010000:1.0:1787826328.164709:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b2b2 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080169b expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.164720:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:1.0:1787826328.164728:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b06d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e50:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080161d expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.164736:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e50:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:1.0:1787826328.164742:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826328.164936:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:15673:x1874663332622848:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 307us (526us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826328.164944:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11130 00000100:00100000:0.0:1787826328.166549:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826328.166623:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332622720 00000100:00100000:3.0:1787826328.166633:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332622720 00000100:00100000:3.0:1787826328.166702:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11131 00000100:00100000:3.0:1787826328.166711:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:34880:x1874663332622720:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826328.166723:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826328.166792:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b10e lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dbc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801640 expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.166880:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbc:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826328.166892:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826328.167457:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:34880:x1874663332622720:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 743us (925us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826328.167469:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11131 00010000:00010000:1.0:1787826328.750494:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826329.514589:0:166442:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:0.0:1787826329.514639:0:166442:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:3.0:1787826329.773845:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826329.835863:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.835910:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.835947:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.835978:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.835999:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.836018:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.836033:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.836055:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.836077:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826329.836130:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332623488 00000100:00100000:0.0:1787826329.836139:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332623488 00000100:00100000:0.0:1787826329.836198:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332624000 00000100:00100000:0.0:1787826329.836202:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332624000 00000100:00100000:0.0:1787826329.836217:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332623360 00000100:00100000:0.0:1787826329.836220:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332623360 00000100:00100000:0.0:1787826329.836235:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332623872 00000100:00100000:0.0:1787826329.836237:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332623872 00000100:00100000:0.0:1787826329.836256:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8833 00000100:00100000:0.0:1787826329.836265:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796eda7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332623488:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826329.836339:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796eda7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332623488:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 74us (389us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826329.836347:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8833 00000100:00100000:0.0:1787826329.836532:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8834 00000100:00100000:0.0:1787826329.836541:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332624000:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826329.836565:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332624000:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (565us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826329.836572:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8834 00000100:00100000:0.0:1787826329.836584:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8835 00000100:00100000:0.0:1787826329.836589:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332623360:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826329.836610:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332623360:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (576us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826329.836615:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8835 00000100:00100000:0.0:1787826329.836625:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8836 00000100:00100000:0.0:1787826329.836631:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332623872:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826329.836652:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332623872:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826329.836657:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8836 00000100:00080000:0.0:1787826329.836673:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787826329.836677:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826329.836871:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332622976 00000100:00100000:2.0:1787826329.836876:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332622976 00000100:00100000:2.0:1787826329.836915:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2228 00000100:00100000:2.0:1787826329.836922:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d68c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332622976:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826329.836957:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d68c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332622976:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 36us (1105us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826329.836963:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2228 00000100:00100000:2.0:1787826329.836993:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332623232 00000100:00100000:2.0:1787826329.836997:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332623232 00000100:00100000:2.0:1787826329.837031:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332623744 00000100:00100000:2.0:1787826329.837034:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332623744 00000100:00100000:2.0:1787826329.837053:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332623104 00000100:00100000:2.0:1787826329.837056:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332623104 00000100:00100000:2.0:1787826329.837071:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332623616 00000100:00100000:2.0:1787826329.837074:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332623616 00000100:00100000:2.0:1787826329.837092:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14318 00000100:00100000:2.0:1787826329.837098:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332623232:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826329.837153:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332623232:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 55us (1243us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826329.837160:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14318 00000100:00100000:2.0:1787826329.837173:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14319 00000100:00100000:2.0:1787826329.837179:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332623744:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826329.837202:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2394:x1874663332623744:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (1224us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826329.837208:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14319 00000100:00100000:2.0:1787826329.837219:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14320 00000100:00100000:2.0:1787826329.837224:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332623104:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826329.837245:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332623104:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 20us (1227us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826329.837251:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14320 00000100:00100000:2.0:1787826329.837262:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14321 00000100:00100000:2.0:1787826329.837268:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332623616:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826329.837289:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332623616:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (1234us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826329.837295:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14321 00000100:00080000:2.0:1787826329.837391:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826330.604316:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346250880:0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.604392:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826330.604450:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787826330.604457:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc30000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346250752:0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.604486:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826330.604531:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346250880 00000100:00100000:0.0:1787826330.604538:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346250880 00000100:00100000:0.0:1787826330.604595:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23809 00000100:00100000:0.0:1787826330.604604:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346250880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.604657:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346250880:12345-0@lo:400:kworker.0 Request processed in 54us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.604665:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23809 00000100:00100000:0.0:1787826330.604737:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346250752 00000100:00100000:0.0:1787826330.604741:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346250752 00000100:00100000:0.0:1787826330.604775:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2229 00000100:00100000:0.0:1787826330.604782:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d6b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346250752:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.604814:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d6b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346250752:12345-0@lo:400:kworker.0 Request processed in 32us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.604821:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2229 00000100:00100000:0.0:1787826330.604861:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc30000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346250752:0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.604874:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346250880:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606348:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346251264:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606404:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.606437:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346251776:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606463:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.606482:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346252288:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606505:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.606541:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1787826330.606547:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346251392:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606572:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.606590:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346251904:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606613:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.606642:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346251136:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606664:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.606680:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346251648:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606699:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.606713:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346252160:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606735:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.606751:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1787826330.606757:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346251008:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.606778:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826330.607013:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346251264 00000100:00100000:0.0:1787826330.607020:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346251264 00000100:00100000:0.0:1787826330.607079:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346252288 00000100:00100000:0.0:1787826330.607083:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346252288 00000100:00100000:0.0:1787826330.607125:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346251136 00000100:00100000:0.0:1787826330.607129:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346251136 00000100:00100000:0.0:1787826330.607151:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346251648 00000100:00100000:0.0:1787826330.607155:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346251648 00000100:00100000:0.0:1787826330.607178:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14322 00000100:00100000:0.0:1787826330.607188:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346251264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607244:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346251264:12345-0@lo:400:kworker.0 Request processed in 58us (840us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.607251:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14322 00000100:00100000:0.0:1787826330.607268:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14323 00000100:00100000:0.0:1787826330.607274:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346252288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607302:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346252288:12345-0@lo:400:kworker.0 Request processed in 28us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.607307:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14323 00000100:00100000:0.0:1787826330.607319:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14324 00000100:00100000:0.0:1787826330.607324:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d814a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346251136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607349:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d814a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346251136:12345-0@lo:400:kworker.0 Request processed in 25us (685us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.607355:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14324 00000100:00100000:0.0:1787826330.607367:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14325 00000100:00100000:0.0:1787826330.607372:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346251648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607398:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346251648:12345-0@lo:400:kworker.0 Request processed in 27us (699us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.607404:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14325 00000100:00100000:0.0:1787826330.607423:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346251776 00000100:00100000:0.0:1787826330.607427:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346251776 00000100:00100000:0.0:1787826330.607451:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346251392 00000100:00100000:0.0:1787826330.607454:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346251392 00000100:00100000:0.0:1787826330.607473:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346252160 00000100:00100000:0.0:1787826330.607475:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346252160 00000100:00100000:0.0:1787826330.607494:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8837 00000100:00100000:0.0:1787826330.607501:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346251776:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607542:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346251776:12345-0@lo:400:kworker.0 Request processed in 41us (1077us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.607548:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8837 00000100:00100000:0.0:1787826330.607560:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8838 00000100:00100000:0.0:1787826330.607566:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346251392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607593:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346251392:12345-0@lo:400:kworker.0 Request processed in 28us (1020us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.607599:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8838 00000100:00100000:0.0:1787826330.607609:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8839 00000100:00100000:0.0:1787826330.607615:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346252160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607642:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346252160:12345-0@lo:400:kworker.0 Request processed in 27us (906us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826330.607647:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8839 00000100:00100000:0.0:1787826330.607700:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346251392:0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607713:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346251264:0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607723:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346251776:0@lo:400:kworker.0 00000100:00100000:0.0:1787826330.607733:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346252288:0@lo:400:kworker.0 00000100:00080000:3.0:1787826330.607946:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826330.607951:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826330.607954:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787826330.607956:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826330.608936:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346251520:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.608982:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.609777:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346252032:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.609824:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826330.609872:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346251136:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.609886:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346251648:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.609897:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346252160:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.609941:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346251008 00000100:00100000:2.0:1787826330.609947:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346251008 00000100:00100000:2.0:1787826330.611625:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346251520 00000100:00100000:2.0:1787826330.611638:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346251520 00000100:00100000:2.0:1787826330.611711:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14326 00000100:00100000:2.0:1787826330.611720:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d814e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346251008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.611808:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d814e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346251008:12345-0@lo:400:kworker.0 Request processed in 88us (5028us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826330.611815:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14326 00000100:00100000:2.0:1787826330.611833:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14327 00000100:00100000:2.0:1787826330.611840:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346251520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.611873:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346251520:12345-0@lo:400:kworker.0 Request processed in 35us (2891us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826330.611883:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14327 00000100:00100000:2.0:1787826330.611910:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346252032 00000100:00100000:2.0:1787826330.611914:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346252032 00000100:00100000:2.0:1787826330.611949:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8840 00000100:00100000:2.0:1787826330.611956:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346252032:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.612003:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346252032:12345-0@lo:400:kworker.0 Request processed in 48us (2179us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826330.612010:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8840 00000100:00100000:2.0:1787826330.612042:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346251904 00000100:00100000:2.0:1787826330.612047:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346251904 00000100:00100000:2.0:1787826330.612074:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23810 00000100:00100000:2.0:1787826330.612081:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346251904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.612141:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346251904:12345-0@lo:400:kworker.0 Request processed in 60us (5526us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826330.612147:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23810 00000100:00080000:2.0:1787826330.612175:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826330.612179:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826330.612214:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346251008:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.612226:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346251520:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.612238:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346252032:0@lo:400:kworker.0 00000100:00100000:2.0:1787826330.612291:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346251904:0@lo:400:kworker.0 00010000:00010000:1.0:1787826330.796467:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826330.854854:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826330.854947:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826330.855529:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332624256 00000100:00100000:3.0:1787826330.855540:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332624256 00000100:00100000:3.0:1787826330.855629:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30846 00000100:00100000:3.0:1787826330.855639:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332624256:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:3.0:1787826330.855750:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332624256:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 112us (802us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826330.855756:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30846 00000100:00080000:0.0:1787826330.856053:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826330.859413:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826330.859465:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826330.860545:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332624512 00000100:00100000:3.0:1787826330.860564:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332624512 00000100:00100000:3.0:1787826330.860630:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30847 00000100:00100000:3.0:1787826330.860639:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332624512:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:3.0:1787826330.860746:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332624512:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 108us (1338us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826330.860753:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30847 00000100:00080000:3.0:1787826330.860777:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826330.865215:0:24159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332624128 00000100:00100000:2.0:1787826330.865229:0:24159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332624128 00002000:00100000:2.0:1787826330.865294:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_010: refresh rw locks for [0x2c0000401:0x521:0x0] (0->5) req@ffff9f7977e0c700 x1874663332624128/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:686/0 lens 496/0 e 0 to 0 dl 1787826341 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826330.865311:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977e0c700 with x1874663332624128 ext(0->5) 00002000:00010000:2.0:1787826330.865317:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7977e0c700 00000100:00100000:3.0:1787826330.867411:0:20179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332624384 00000100:00100000:3.0:1787826330.867426:0:20179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332624384 00002000:00100000:3.0:1787826330.867482:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000400:0x4ce:0x0] (0->5) req@ffff9f7847704a80 x1874663332624384/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:686/0 lens 496/0 e 0 to 0 dl 1787826341 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826330.867493:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7847704a80 with x1874663332624384 ext(0->5) 00002000:00010000:3.0:1787826330.867500:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7847704a80 00000100:00100000:3.0:1787826330.867542:0:20179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27653 00000100:00100000:3.0:1787826330.867554:0:20179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332624128:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826330.870640:0:20179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2393:x1874663332624128:12345-192.168.203.56@tcp:4:bash.0 Request processed in 3077us (15783us total) trans 4294976161 rc 0/0 00000100:00100000:3.0:1787826330.870671:0:20179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27653 00002000:00100000:3.0:1787826330.870685:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x521:0x0] (0->5) req@ffff9f7977e0c700 x1874663332624128/t4294976161(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:686/0 lens 496/448 e 0 to 0 dl 1787826341 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826330.870696:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977e0c700 with x1874663332624128 ext(0->5) 00002000:00010000:3.0:1787826330.870702:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7977e0c700 00000100:00100000:3.0:1787826330.870734:0:20179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27654 00000100:00100000:3.0:1787826330.870748:0:20179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332624384:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826330.871754:0:20179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332624384:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1006us (12283us total) trans 4294973853 rc 0/0 00000100:00100000:3.0:1787826330.871762:0:20179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27654 00002000:00100000:3.0:1787826330.871770:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000400:0x4ce:0x0] (0->5) req@ffff9f7847704a80 x1874663332624384/t4294973853(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:686/0 lens 496/448 e 0 to 0 dl 1787826341 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826330.871779:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7847704a80 with x1874663332624384 ext(0->5) 00002000:00010000:3.0:1787826330.871784:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7847704a80 00000001:00080000:2.0:1787826330.875878:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976161 is committed 00000100:00100000:1.0:1787826331.116798:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d817100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346252416:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826331.116880:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1787826331.116964:0:9993:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8143328857616ns, next at 8144328854689ns, rc = 0 00000100:00100000:3.0:1787826331.117461:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346252416 00000100:00100000:3.0:1787826331.117471:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346252416 00000100:00100000:3.0:1787826331.117535:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6239 00000100:00100000:3.0:1787826331.117545:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346252416:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826331.117569:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914711040 avail 3683131392 00000004:00080000:0.0:1787826331.120658:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 405 committed 00010000:00010000:0.0:1787826331.120709:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b1af expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826331.120725:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b1af expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.120772:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1af expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.120790:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826331.120821:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4b1a8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b1af expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826331.120841:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985426 is committed 00000020:00080000:0.0:1787826331.120859:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78500b1840 refcount 2 committed 0 result 0 batchid 3425 00000020:00080000:0.0:1787826331.120863:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f785042f400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78453d9900 00000020:00080000:0.0:1787826331.120867:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1529 00000020:00080000:0.0:1787826331.120869:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f785042f300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a5f3a00 00000020:00080000:0.0:1787826331.120872:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1210 00000001:00080000:0.0:1787826331.120891:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985423 is committed 00000020:00080000:0.0:1787826331.120895:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78453cec00 refcount 2 committed 0 result 0 batchid 3424 00000020:00080000:0.0:1787826331.120898:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78453cef00 00000020:00080000:0.0:1787826331.120900:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1528 00000020:00080000:0.0:1787826331.120902:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629dd00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f79616c5000 00000020:00080000:0.0:1787826331.120904:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1209 00000004:00080000:0.0:1787826331.120911:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 404 committed 00000020:00080000:0.0:1787826331.120915:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784f58bcc0 refcount 2 committed 0 result 0 batchid 3423 00000020:00080000:0.0:1787826331.120918:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f58b840 00000020:00080000:0.0:1787826331.120921:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1527 00000020:00080000:0.0:1787826331.120922:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78525e4c00 00000020:00080000:0.0:1787826331.120925:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1208 00000020:00080000:0.0:1787826331.120931:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784f58b6c0 refcount 2 committed 0 result 0 batchid 3422 00000020:00080000:0.0:1787826331.120935:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f58bd80 00000020:00080000:0.0:1787826331.120937:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1526 00000020:00080000:0.0:1787826331.120939:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78525e5c00 00000020:00080000:0.0:1787826331.120942:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1207 00000001:00080000:0.0:1787826331.120947:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985417 is committed 00000020:00080000:0.0:1787826331.120964:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3421 batchid updated 00000020:00080000:0.0:1787826331.120991:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3425 committed batchid 3421 00000004:00080000:0.0:1787826331.121154:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784dedd300 with version 6848 00000020:00080000:0.0:1787826331.121164:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3418 cancel update log [0x200000401:0x3:0x0].1521: rc = 0 00000020:00080000:0.0:1787826331.122899:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3418 cancel update log [0x240000400:0x3:0x0].1203: rc = 0 00000004:00080000:0.0:1787826331.122957:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784dedd540 with version 6849 00000020:00080000:0.0:1787826331.122961:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3419 cancel update log [0x200000401:0x3:0x0].1522: rc = 0 00000020:00080000:0.0:1787826331.122999:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3419 cancel update log [0x240000400:0x3:0x0].1204: rc = 0 00000004:00080000:0.0:1787826331.123022:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784deddc00 with version 6850 00000020:00080000:0.0:1787826331.123025:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3420 cancel update log [0x200000401:0x3:0x0].1523: rc = 0 00000020:00080000:0.0:1787826331.123043:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3420 cancel update log [0x240000400:0x3:0x0].1205: rc = 0 00000004:00080000:0.0:1787826331.123064:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784dedd840 with version 6851 00000020:00080000:0.0:1787826331.123066:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3421 cancel update log [0x200000401:0x3:0x0].1525: rc = 0 00000020:00080000:0.0:1787826331.123088:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3421 cancel update log [0x240000400:0x3:0x0].1206: rc = 0 00000020:00080000:0.0:1787826331.123097:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3421 00000004:00080000:0.0:1787826331.123124:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6848 rpc_version 6848 00000020:00100000:0.0:1787826331.123144:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826331.123155:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7847707b80 x1874663346252928/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787826331.123384:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123395:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123404:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123410:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826331.123416:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123425:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123441:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:2.0:1787826331.123444:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826331.123450:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123456:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b265 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123482:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123489:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123494:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123498:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826331.123501:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123506:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123524:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:2.0:1787826331.123527:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826331.123530:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123535:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b26c lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123556:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123563:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123569:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123574:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826331.123577:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123583:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123590:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x4ec94fd6 (ffff9f79785cce00) 00010000:00010000:2.0:1787826331.123592:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826331.123595:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.123600:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b30d lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826331.123634:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346252672:0@lo:6:osp-syn-1-1.0 00000100:00100000:2.0:1787826331.123714:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826331.123761:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826331.123768:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346252544:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826331.123920:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346252928:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826331.123957:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826331.123976:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826331.123999:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346252672 00000100:00100000:0.0:1787826331.124006:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346252672 00000100:00100000:0.0:1787826331.124050:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30848 00000100:00100000:0.0:1787826331.124059:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346252672:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1787826331.124085:0:8443:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1227 count 1 00010000:00010000:0.0:1787826331.124128:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.124150:0:8443:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.124164:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4cb:0x0].0x0 since=9s v=(0 0) 00010000:00010000:0.0:1787826331.124168:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cb:0x0].0x0 since=0s v=(1 0) 00000100:00100000:2.0:1787826331.124269:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826331.124310:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346252544 00000100:00100000:2.0:1787826331.124315:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346252544 00000100:00100000:2.0:1787826331.124353:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11132 00000100:00100000:2.0:1787826331.124360:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346252544:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826331.124369:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826331.124387:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b1af lrc: 0/0,0 mode: --/PW res: [0x200000401:0x194:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b1a8 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.124399:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x194:0x0].0x5a734ce9 (ffff9f79785cc700) 00010000:00010000:2.0:1787826331.124406:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826331.124441:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346252544:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 82us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826331.124449:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11132 00000100:00100000:2.0:1787826331.124490:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346252544:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787826331.124568:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346252928 00000100:00100000:2.0:1787826331.124573:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346252928 00000100:00100000:2.0:1787826331.124601:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23811 00000100:00100000:2.0:1787826331.124608:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346252928:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826331.124748:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346252928:12345-0@lo:1000:osp_up0-1.0 Request processed in 140us (789us total) trans 8589957419 rc 0/0 00000100:00100000:2.0:1787826331.124754:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23811 00010000:00010000:0.0:1787826331.124944:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.124977:0:8443:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 6 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.124991:0:8443:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 6 pid: 8759 timeout: 8243 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826331.125003:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8443:1874663346253056:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826331.125083:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826331.125142:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346252928:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826331.125153:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6849 rpc_version 6849 00000020:00100000:0.0:1787826331.125170:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826331.126567:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7847704e00 x1874663346253312/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826331.126596:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346253312:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826331.126662:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826331.126697:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826331.126727:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346253312 00000100:00100000:0.0:1787826331.126734:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346253312 00000100:00100000:0.0:1787826331.126791:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23812 00000100:00100000:0.0:1787826331.126801:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346253312:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826331.126951:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346253312:12345-0@lo:1000:osp_up0-1.0 Request processed in 150us (288us total) trans 8589957420 rc 0/0 00000100:00100000:0.0:1787826331.126958:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23812 00000100:00100000:0.0:1787826331.127007:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346253312:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826331.127018:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6850 rpc_version 6850 00000020:00100000:0.0:1787826331.127035:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826331.127039:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f785189f100 x1874663346253568/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826331.127050:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346253568:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826331.127076:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826331.127091:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784add8600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826331.127130:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346253568 00000100:00100000:0.0:1787826331.127134:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346253568 00000100:00100000:0.0:1787826331.127161:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23813 00000100:00100000:0.0:1787826331.127168:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346253568:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826331.128790:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3655:x1874663346252416:12345-0@lo:13:osp-pre-1-1.0 Request processed in 11228us (11890us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826331.128805:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6239 00000100:00100000:0.0:1787826331.129617:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346253568:12345-0@lo:1000:osp_up0-1.0 Request processed in 2447us (2537us total) trans 8589957421 rc 0/0 00000100:00100000:0.0:1787826331.129627:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23813 00000100:00100000:2.0:1787826331.130407:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346253568:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826331.130427:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6851 rpc_version 6851 00000020:00100000:2.0:1787826331.130453:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826331.130459:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d815c00 x1874663346253824/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826331.130481:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346253824:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826331.130545:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826331.130577:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826331.131006:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346253824 00000100:00100000:0.0:1787826331.131015:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346253824 00000100:00100000:0.0:1787826331.131070:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23814 00000100:00100000:0.0:1787826331.131080:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346253824:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826331.131520:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346253824:12345-0@lo:1000:osp_up0-1.0 Request processed in 430us (963us total) trans 8589957422 rc 0/0 00000100:00100000:0.0:1787826331.131528:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23814 00000100:00100000:2.0:1787826331.131611:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d815c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346253824:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826331.131697:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d817100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346252416:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826331.136622:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826331.136735:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8443:1874663346253056:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826331.136775:0:8443:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826331.137021:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332624640 00000100:00100000:1.0:1787826331.137029:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332624640 00010000:00010000:1.0:1787826331.137071:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 7 pid: 8759 timeout: 8243 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826331.137151:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21984 00000100:00100000:1.0:1787826331.137160:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332624640:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826331.137182:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826331.137188:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 7 pid: 8759 timeout: 8243 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826331.137199:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 7 pid: 8759 timeout: 8243 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826331.137221:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009eda5395/0xe21309aa81f4afc5 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608015de expref: 7 pid: 8759 timeout: 8243 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826331.137236:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826331.137240:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826331.137246:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826331.137256:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826331.137281:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826331.137466:0:8443:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.137476:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.137487:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.137495:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826331.137520:0:8443:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826331.137736:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346252672:12345-0@lo:6:osp-syn-1-1.0 Request processed in 13677us (14020us total) trans 4294976162 rc 0/0 00000100:00100000:2.0:1787826331.137745:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30848 00010000:00010000:0.0:1787826331.138128:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826331.138137:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.138146:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826331.138150:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.138158:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.138181:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.138189:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e08490e4/0xe21309aa81f4b329 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4cb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826331.138209:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826331.138211:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826331.138213:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1787826331.138324:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784d816a00/2, rc 0, transno 4294976162 00000100:00100000:2.0:1787826331.138331:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346252672:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1787826331.138519:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332624640:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1349us (1896us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826331.138529:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21984 00000100:00100000:3.0:1787826331.628912:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346253952:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826331.628992:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826331.629421:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826331.629433:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346254080:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826331.629466:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826331.629523:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346254080 00000100:00100000:3.0:1787826331.629530:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346254080 00000100:00100000:3.0:1787826331.629582:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23815 00000100:00100000:3.0:1787826331.629592:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346254080:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787826331.629631:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442787328 avail 1296547840 00000100:00100000:3.0:1787826331.629681:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346254080:12345-0@lo:41:osp-pre-0-1.0 Request processed in 90us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826331.629697:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23815 00000100:00100000:3.0:1787826331.629747:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346254080:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826331.630288:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826331.630300:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346254464:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826331.630341:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826331.630380:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346253952 00000100:00100000:1.0:1787826331.630385:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346253952 00000100:00100000:1.0:1787826331.630417:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346254464 00000100:00100000:1.0:1787826331.630420:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346254464 00000100:00100000:1.0:1787826331.630434:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6240 00000100:00100000:1.0:1787826331.630441:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346253952:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826331.630457:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826331.630505:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346253952:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (1513us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826331.630521:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6240 00000100:00100000:1.0:1787826331.630535:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6241 00000100:00100000:1.0:1787826331.630542:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346254464:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826331.630553:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914711040 avail 3683131392 00000100:00100000:1.0:1787826331.630588:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346254464:12345-0@lo:13:osp-pre-1-0.0 Request processed in 47us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826331.630594:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6241 00000100:00100000:1.0:1787826331.630611:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346254336:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826331.630638:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826331.630657:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826331.630663:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346254208:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826331.630699:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826331.630734:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346254336 00000100:00100000:1.0:1787826331.630738:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346254336 00000100:00100000:1.0:1787826331.630758:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6242 00000100:00100000:1.0:1787826331.630765:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346254336:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826331.630774:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826331.631102:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346254464:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826331.631176:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346253952:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826331.631281:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346254336:12345-0@lo:13:osp-pre-0-1.0 Request processed in 515us (641us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826331.631288:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6242 00000100:00100000:0.0:1787826331.631591:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346254336:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826331.631627:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346254208 00000100:00100000:0.0:1787826331.631632:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346254208 00000100:00100000:0.0:1787826331.631671:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23816 00000100:00100000:0.0:1787826331.631679:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346254208:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1787826331.631717:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443385344 avail 1297145856 00000100:00100000:0.0:1787826331.631775:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346254208:12345-0@lo:41:osp-pre-1-0.0 Request processed in 96us (1084us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826331.631781:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23816 00000100:00100000:0.0:1787826331.631818:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985417 gen 2 00000004:00080000:0.0:1787826331.631824:0:3653:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985416 committed transno 4294985417 00000020:00080000:0.0:1787826331.631831:0:3653:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7845dd19c0 refcount 2 committed 0 result 0 batchid 4294979084 00000020:00080000:0.0:1787826331.631835:0:3653:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d000 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7845dd1240 00000020:00080000:0.0:1787826331.631839:0:3653:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].940 00000020:00080000:0.0:1787826331.631841:0:3653:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784c509e00 00000020:00080000:0.0:1787826331.631845:0:3653:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].937 00000004:00080000:0.0:1787826331.632755:0:3653:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985417 committed transno 4294985417 00000100:00100000:0.0:1787826331.632810:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346254208:0@lo:41:osp-pre-1-0.0 00000020:00080000:0.0:1787826331.632841:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979084 committed batchid 4294979083 00000020:00080000:0.0:1787826331.633091:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979083 00000001:00080000:3.0:1787826331.638498:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973853 is committed 00000001:00080000:3.0:1787826331.638527:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973852 is committed 00010000:00010000:1.0:1787826331.824275:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826332.103982:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826332.104584:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332624768 00000100:00100000:0.0:1787826332.104595:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332624768 00000100:00080000:1.0:1787826332.104739:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826332.104904:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68507 00000100:00100000:0.0:1787826332.104918:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225555:x1874663332624768:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826332.104941:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826332.104969:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b330 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016cc expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.105049:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.105063:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.105075:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826332.105157:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826332.105161:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826332.105182:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.105209:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4b330 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.105226:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.105234:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826332.105237:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826332.105239:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826332.105306:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225555:x1874663332624768:12345-192.168.203.56@tcp:101:cat.0 Request processed in 392us (1334us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.105314:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68507 00000100:00100000:2.0:1787826332.109547:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826332.112586:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332624896 00000100:00100000:1.0:1787826332.112603:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332624896 00000100:00100000:0.0:1787826332.112729:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68508 00000100:00100000:0.0:1787826332.112741:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225555:x1874663332624896:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826332.112762:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826332.112796:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b33e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016d3 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.112996:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113012:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113034:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113052:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113060:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113067:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113176:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4b353 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113184:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4b353 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113191:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4b353 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826332.113213:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4b353 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113228:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113235:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113241:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113248:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113254:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.113258:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113265:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113277:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826332.113280:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826332.113285:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113291:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b345 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113303:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113309:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113315:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113320:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.113323:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113329:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113337:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784b4a5400) 00010000:00010000:0.0:1787826332.113339:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826332.113343:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113349:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b34c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113586:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b33e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016d3 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113601:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4b353 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016d3 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.113609:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826332.113611:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826332.113613:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826332.113687:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225555:x1874663332624896:12345-192.168.203.56@tcp:101:cat.0 Request processed in 949us (4227us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.113696:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68508 00000100:00100000:1.0:1787826332.116733:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826332.116836:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332625024 00000100:00100000:0.0:1787826332.116847:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332625024 00000100:00100000:0.0:1787826332.116925:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68509 00000100:00100000:0.0:1787826332.116937:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:225555:x1874663332625024:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826332.116958:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826332.116982:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b35a lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016da expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.117059:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b361 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.117073:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b361 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.117086:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b361 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826332.117208:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e52:0x0]: returning size 4096 00000004:00200000:0.0:1787826332.117214:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826332.117255:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b35a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016da expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.117273:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b361 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016da expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.117282:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:0.0:1787826332.117285:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826332.117288:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826332.117348:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225555:x1874663332625024:12345-192.168.203.56@tcp:101:cat.0 Request processed in 413us (622us total) trans 0 rc 0/0 00000100:00080000:1.0:1787826332.117640:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826332.117757:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68509 00000100:00100000:3.0:1787826332.121861:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826332.123258:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332625152 00000100:00100000:2.0:1787826332.123271:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332625152 00000100:00100000:2.0:1787826332.125459:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68510 00000100:00100000:2.0:1787826332.125489:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225555:x1874663332625152:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826332.125520:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826332.125550:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4b368 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016e1 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.125626:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.125640:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.125651:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826332.125702:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x197:0x0]: returning size 4096 00000004:00200000:2.0:1787826332.125707:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826332.125745:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4b368 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.125763:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.125784:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f7961f9a100) 00010000:00010000:2.0:1787826332.125787:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826332.125791:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826332.126075:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225555:x1874663332625152:12345-192.168.203.56@tcp:101:cat.0 Request processed in 593us (4223us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826332.126083:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68510 00000100:00100000:3.0:1787826332.136722:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826332.139485:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332625280 00000100:00100000:2.0:1787826332.139500:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332625280 00000100:00100000:2.0:1787826332.139576:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68511 00000100:00100000:2.0:1787826332.139587:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225555:x1874663332625280:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826332.139611:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826332.139636:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b376 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016e8 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.139700:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.139714:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.139724:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826332.139776:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x195:0x0]: returning size 4096 00000004:00200000:2.0:1787826332.139782:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826332.139944:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b376 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.139963:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.139972:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a900) 00010000:00010000:2.0:1787826332.139975:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826332.139978:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826332.140045:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225555:x1874663332625280:12345-192.168.203.56@tcp:101:cat.0 Request processed in 460us (3332us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826332.140052:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68511 00000100:00100000:1.0:1787826332.149391:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826332.149608:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332625536 00000100:00100000:0.0:1787826332.149618:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332625536 00000100:00100000:0.0:1787826332.149679:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68512 00000100:00100000:0.0:1787826332.149690:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225555:x1874663332625536:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826332.149709:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826332.149731:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b384 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016ef expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.149879:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.149894:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.149905:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.149916:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.149923:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.149930:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:0.0:1787826332.150004:0:24702:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000100:00100000:3.0:1787826332.622361:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826332.622737:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332625664 00000100:00100000:2.0:1787826332.622748:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332625664 00000100:00100000:2.0:1787826332.622845:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68513 00000100:00100000:2.0:1787826332.622857:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225557:x1874663332625664:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826332.622888:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826332.622913:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b399 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016f6 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.622982:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.622995:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.623006:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826332.623062:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826332.623067:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826332.623099:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.623305:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b399 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.623326:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.623334:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826332.623337:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826332.623340:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826332.623479:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225557:x1874663332625664:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 624us (1139us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826332.623487:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68513 00000100:00080000:1.0:1787826332.624853:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826332.626409:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826332.626638:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332625792 00000100:00100000:2.0:1787826332.626649:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332625792 00000100:00100000:2.0:1787826332.626762:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68514 00000100:00100000:2.0:1787826332.626782:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225557:x1874663332625792:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826332.626805:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826332.626827:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3a7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608016fd expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.626899:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.626914:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.626925:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.626941:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.626948:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.626955:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627043:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627058:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627065:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826332.627088:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627130:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627138:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627144:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627150:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627156:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826332.627160:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627167:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627179:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826332.627182:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826332.627187:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627193:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b3ae lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627209:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627215:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627220:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627224:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826332.627227:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627233:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627240:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961f9a200) 00010000:00010000:2.0:1787826332.627243:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826332.627246:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627252:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3b5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627287:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3a7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627301:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.627309:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:2.0:1787826332.627312:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826332.627314:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826332.627454:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225557:x1874663332625792:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 682us (1054us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826332.627461:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68514 00000100:00100000:0.0:1787826332.632358:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826332.639102:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332625920 00000100:00100000:2.0:1787826332.639139:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332625920 00000100:00100000:2.0:1787826332.639232:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68515 00000100:00100000:2.0:1787826332.639243:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:225557:x1874663332625920:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826332.639264:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826332.639286:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b3c3 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801704 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.639354:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b3ca lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.639367:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b3ca lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826332.639572:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1787826332.641033:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b3ca lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826332.641295:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e52:0x0]: returning size 4096 00000004:00200000:2.0:1787826332.641302:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826332.641359:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b3c3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801704 expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.641621:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b3ca lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801704 expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.641632:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:2.0:1787826332.641636:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826332.641640:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826332.641728:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225557:x1874663332625920:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2485us (9380us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826332.641737:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68515 00000100:00100000:2.0:1787826332.650692:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826332.650790:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332626048 00000100:00100000:0.0:1787826332.650801:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332626048 00000100:00100000:0.0:1787826332.650866:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68516 00000100:00100000:0.0:1787826332.650878:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225557:x1874663332626048:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826332.650900:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826332.650923:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4b3d1 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080170b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651010:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651023:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651034:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651050:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 3/1,0 mode: --/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651057:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651064:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651186:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b3e6 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651194:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b3e6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651199:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b3e6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826332.651242:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e53:0x0]: returning size 6 00000004:00200000:0.0:1787826332.651246:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826332.651266:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b3e6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651277:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b3e6 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651286:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651293:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651299:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651305:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651311:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.651316:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651322:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651333:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f7961f9a100) 00010000:00010000:0.0:1787826332.651335:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826332.651339:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651348:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b3d8 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651361:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651366:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651372:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651376:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.651378:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651383:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651389:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x4ec94fd6 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826332.651392:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826332.651394:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 2/0,0 mode: --/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651398:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b3df lrc: 0/0,0 mode: --/PR res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651429:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4b3d1 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080170b expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651448:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b3e6 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080170b expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.651455:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:0.0:1787826332.651458:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826332.651460:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826332.651542:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225557:x1874663332626048:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 666us (860us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.651549:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68516 00000100:00100000:1.0:1787826332.665786:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826332.665898:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332626176 00000100:00100000:3.0:1787826332.665908:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332626176 00000100:00100000:3.0:1787826332.665987:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68517 00000100:00100000:3.0:1787826332.666427:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225557:x1874663332626176:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826332.666562:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826332.666590:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b3ed lrc: 2/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801712 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666665:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666680:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666692:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666704:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666712:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666724:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666820:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b402 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666828:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b402 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666836:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b402 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826332.666880:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbd:0x0]: returning size 6 00000004:00200000:2.0:1787826332.666885:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826332.666905:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b402 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666916:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b402 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666926:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666932:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666939:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666946:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666955:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826332.666960:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666967:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666980:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a900) 00010000:00010000:2.0:1787826332.666982:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826332.666986:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.666992:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b3f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667087:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667094:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667100:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667221:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826332.667224:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667231:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667242:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:2.0:1787826332.667245:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826332.667249:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667255:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b3fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667287:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b3ed lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801712 expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667300:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b402 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801712 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.667307:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f7961f9a200) 00010000:00010000:2.0:1787826332.667309:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826332.667312:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:2.0:1787826332.667493:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225557:x1874663332626176:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1493us (1716us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826332.667510:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68517 00000100:00100000:3.0:1787826332.677863:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826332.681792:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826332.681899:0:6542:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332626560 00000100:00100000:1.0:1787826332.681910:0:6542:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332626560 00000100:00100000:1.0:1787826332.681982:0:6542:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1232 00000100:00100000:1.0:1787826332.681995:0:6542:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225557:x1874663332626560:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:1.0:1787826332.682060:0:6542:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826332.682091:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b402 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801712 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.682129:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f7961f9a200) 00010000:00010000:1.0:1787826332.682143:0:6542:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826332.682224:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346254592:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.682293:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826332.682330:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.682362:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346254592 00000100:00100000:3.0:1787826332.682367:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346254592 00000100:00100000:3.0:1787826332.682420:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23817 00000100:00100000:3.0:1787826332.682428:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346254592:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.682521:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346254592:12345-0@lo:1000:mdt_io00_002.0 Request processed in 94us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826332.682527:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23817 00000100:00100000:1.0:1787826332.682724:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346254592:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.682800:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346254720:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.682827:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826332.682853:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826332.682903:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346254720 00000100:00100000:0.0:1787826332.682908:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346254720 00000100:00100000:0.0:1787826332.682954:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23818 00000100:00100000:0.0:1787826332.682962:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346254720:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.683049:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346254720:12345-0@lo:1000:mdt_io00_002.0 Request processed in 87us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.683055:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23818 00000100:00100000:3.0:1787826332.683288:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346254720:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826332.683316:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x197:0x0] 00000100:00100000:3.0:1787826332.683346:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346254848:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.683387:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.683403:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826332.683760:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346254848 00000100:00100000:0.0:1787826332.683766:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346254848 00000100:00100000:0.0:1787826332.683807:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23819 00000100:00100000:0.0:1787826332.683816:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346254848:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.683889:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346254848:12345-0@lo:1000:mdt_io00_002.0 Request processed in 74us (515us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.683896:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23819 00000100:00100000:0.0:1787826332.683943:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346254848:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826332.683969:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e52:0x0] 00000100:00100000:0.0:1787826332.684004:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346254976:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.684047:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826332.684063:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f37600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826332.684076:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346254976 00000100:00100000:0.0:1787826332.684079:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346254976 00000100:00100000:0.0:1787826332.684132:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23820 00000100:00100000:0.0:1787826332.684140:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346254976:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.684193:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346254976:12345-0@lo:1000:mdt_io00_002.0 Request processed in 53us (145us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.684200:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23820 00000100:00100000:0.0:1787826332.684228:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346254976:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826332.684263:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332626304 00000100:00100000:2.0:1787826332.684270:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332626304 00000004:00080000:0.0:1787826332.684273:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x197:0x0] 00000100:00100000:0.0:1787826332.684302:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346255104:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826332.684322:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11133 00000100:00100000:0.0:1787826332.684335:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826332.684336:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332626304:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:2.0:1787826332.684349:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787826332.684354:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f37000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826332.684369:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346255104 00010000:00010000:2.0:1787826332.684374:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b3e6 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080170b expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.684397:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f784b4a5100) 00010000:00010000:2.0:1787826332.684408:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826332.684453:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332626304:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 122us (2671us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826332.684460:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11133 00000100:00100000:0.0:1787826332.685796:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346255104 00000100:00100000:0.0:1787826332.685914:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23821 00000100:00100000:0.0:1787826332.685923:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346255104:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.686099:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346255104:12345-0@lo:1000:mdt_io00_002.0 Request processed in 176us (1767us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.686120:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23821 00000100:00100000:2.0:1787826332.686226:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346255104:0@lo:1000:mdt_io00_002.0 00000004:00020000:2.0:1787826332.686281:0:6542:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00000004:00020000:0.0:1787826332.686333:0:24702:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4463 00010000:00010000:2.0:1787826332.686340:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.686352:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.686361:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.686381:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346255232:0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826332.686391:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686411:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.686420:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826332.686425:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826332.686438:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbd:0x0]: returning size 6 00000100:00100000:2.0:1787826332.686445:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58bf00 going to sleep for 11 seconds 00010000:00010000:0.0:1787826332.686535:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686543:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686550:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686557:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686562:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.686567:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686574:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686585:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a900) 00010000:00010000:0.0:1787826332.686588:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826332.686592:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 3/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686603:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b38b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686615:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686621:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686627:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686632:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.686635:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686641:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686646:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826332.686649:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826332.686651:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686657:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b392 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686701:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b384 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686719:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.686726:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826332.686728:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826332.686731:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826332.686782:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225555:x1874663332625536:12345-192.168.203.56@tcp:101:cat.0 Request processed in 537095us (537402us total) trans 8589957423 rc 0/0 00000100:00100000:0.0:1787826332.686789:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68512 00000100:00100000:3.0:1787826332.686893:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346255232 00000100:00100000:3.0:1787826332.686898:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346255232 00000100:00100000:3.0:1787826332.686929:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23822 00000100:00100000:3.0:1787826332.686936:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346255232:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:3.0:1787826332.686946:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826332.686961:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b41e lrc: 2/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b409 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.686979:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.686989:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b41e lrc: 4/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b409 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.687085:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 6 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.687097:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 6 pid: 35098 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826332.687125:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346255360:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826332.687169:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.697171:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346255360:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826332.697406:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826332.697456:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826332.697551:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332626688 00000100:00100000:2.0:1787826332.697559:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332626688 00010000:00010000:2.0:1787826332.697599:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 7 pid: 35098 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.697643:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21985 00000100:00100000:2.0:1787826332.697653:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2396:x1874663332626688:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826332.697667:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826332.697675:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 7 pid: 35098 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.697686:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 7 pid: 35098 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.697707:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b36f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e1 expref: 7 pid: 35098 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.697725:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f7961f9a100) 00010000:00010000:2.0:1787826332.697732:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b41e lrc: 3/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b409 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.697743:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b41e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b409 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.697751:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b41e lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b409 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.697784:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b41e lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b409 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.697800:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6511:1874663346255488:0@lo:105: 00000100:00100000:2.0:1787826332.697845:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826332.697881:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.697981:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332626816 00000100:00100000:1.0:1787826332.697986:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332626816 00000100:00100000:1.0:1787826332.698290:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30849 00000100:00100000:1.0:1787826332.698300:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332626816:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826332.698315:0:8444:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826332.698332:0:8444:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4b425 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801719 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826332.698374:0:8444:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8444:1874663346255616:192.168.203.56@tcp:106: 00000100:00100000:1.0:1787826332.698440:0:8444:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.698456:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346255488 00000100:00100000:1.0:1787826332.698459:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346255488 00000100:00100000:1.0:1787826332.698489:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7884 00000100:00100000:1.0:1787826332.698496:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6511:x1874663346255488:12345-0@lo:105: 00010000:00010000:1.0:1787826332.698517:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.698527:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.698535:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.698550:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.698557:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.698565:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.698577:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eee00) 00000100:00100000:1.0:1787826332.698623:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6511:x1874663346255488:12345-0@lo:105: Request processed in 128us (777us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.698630:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7884 00000100:00100000:1.0:1787826332.698668:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6511:1874663346255488:0@lo:105: 00010000:00010000:1.0:1787826332.698685:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826332.698722:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332626688:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1070us (1734us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.698729:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21985 00010000:00010000:3.0:1787826332.699059:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b41e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b409 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.699070:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f7961f9a100) 00010000:00010000:3.0:1787826332.699074:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826332.699076:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826332.699149:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346255232:12345-0@lo:101:mdt_io00_002.0 Request processed in 12214us (12730us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826332.699156:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23822 00000100:00100000:1.0:1787826332.699402:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346255232:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826332.699419:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826332.699441:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x197:0x0] 00010000:00010000:1.0:1787826332.699476:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.699489:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.699497:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826332.699524:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346255744:0@lo:101:mdt_io00_002.0 00000100:00100000:1.0:1787826332.699555:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826332.699585:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.699769:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346255744 00000100:00100000:3.0:1787826332.699776:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346255744 00000100:00100000:3.0:1787826332.699818:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23823 00000100:00100000:3.0:1787826332.699826:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346255744:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:3.0:1787826332.699838:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826332.699853:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4b433 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b42c expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.699878:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4b433 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b42c expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.699891:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4b433 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b42c expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.699898:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x4ec94fd6 (ffff9f79785cc800) 00010000:00010000:3.0:1787826332.699900:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826332.699903:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826332.699941:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346255744:12345-0@lo:101:mdt_io00_002.0 Request processed in 115us (385us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826332.699947:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23823 00000100:00100000:2.0:1787826332.700019:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346255744:0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787826332.700032:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b433 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.700046:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b433 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826332.700060:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x197:0x0] 00010000:00010000:2.0:1787826332.700078:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.700090:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.700097:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 4/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.700149:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.700161:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 10 pid: 11629 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.700172:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346255872:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826332.700222:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58bf00 going to sleep for 11 seconds 00000100:00080000:0.0:1787826332.700589:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:1.0:1787826332.701570:0:8444:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x2c0000401:0x521:0x0] updating lvb size: 0 -> 6 00002000:00010000:1.0:1787826332.701577:0:8444:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x2c0000401:0x521:0x0] updating lvb mtime: 0 -> 1787826327 00002000:00010000:1.0:1787826332.701580:0:8444:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x2c0000401:0x521:0x0] updating lvb atime: 0 -> 1787826327 00002000:00010000:1.0:1787826332.701582:0:8444:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x2c0000401:0x521:0x0] updating lvb ctime: 0 -> 1787826327 00002000:00010000:1.0:1787826332.701583:0:8444:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x2c0000401:0x521:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:1.0:1787826332.701606:0:8444:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8444:1874663346255616:192.168.203.56@tcp:106: 00010000:00010000:1.0:1787826332.701640:0:8444:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4b425 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801719 expref: 6 pid: 8444 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.701660:0:8444:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4b425 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801719 expref: 6 pid: 8444 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.701674:0:8444:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826332.701734:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332626816:12345-192.168.203.56@tcp:101:cat.0 Request processed in 3434us (4276us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826332.701742:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30849 00000100:00100000:1.0:1787826332.703341:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346255872:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826332.703387:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826332.706206:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826332.706277:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332626944 00000100:00100000:2.0:1787826332.706286:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332626944 00000100:00100000:2.0:1787826332.706351:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30850 00000100:00100000:2.0:1787826332.706362:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:225555:x1874663332626944:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826332.706391:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826332.706412:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801720 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.706454:0:35096:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.706463:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x521:0x0].0x0 since=4s v=(0 0) 00010000:00010000:2.0:1787826332.706468:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826332.706474:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.706519:0:35096:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.706532:0:35096:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 5 pid: 35096 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.706547:0:35096:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:35096:1874663346256000:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826332.706635:0:35096:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.708460:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826332.708662:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332627072 00000100:00100000:1.0:1787826332.708671:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332627072 00010000:00010000:1.0:1787826332.708715:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 11 pid: 11629 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826332.708754:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21986 00000100:00100000:1.0:1787826332.708764:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332627072:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826332.708788:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826332.708796:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 11 pid: 11629 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.708809:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 11 pid: 11629 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.708832:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b37d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016e8 expref: 11 pid: 11629 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.708851:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a900) 00010000:00010000:1.0:1787826332.708856:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.708865:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.708871:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.709085:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826332.709179:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332627072:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 415us (727us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.709187:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21986 00010000:00010000:2.0:1787826332.709620:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.709631:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.709652:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.709664:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.709673:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.709741:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256128:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826332.709793:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826332.709872:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.709905:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826332.710042:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332627200 00000100:00100000:2.0:1787826332.710049:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332627200 00010000:00010000:2.0:1787826332.710088:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 6 pid: 35096 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.710135:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21987 00000100:00100000:2.0:1787826332.710144:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332627200:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826332.710155:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826332.710176:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 6 pid: 35096 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.710188:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 6 pid: 35096 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.710208:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826332.710213:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826332.710221:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.710231:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.710259:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826332.710270:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:17812:1874663346256256:192.168.203.56@tcp:105: 00010000:00010000:3.0:1787826332.710562:0:35096:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000632eea58/0xe21309aa81f4b2f8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016b7 expref: 6 pid: 35096 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826332.710577:0:35096:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:35096:1874663346256000:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826332.710602:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.710611:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:3.0:1787826332.710703:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225555:x1874663332626944:12345-192.168.203.56@tcp:101:cat.0 Request processed in 4342us (4506us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826332.710710:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30850 00000100:00100000:0.0:1787826332.710854:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346256128 00000100:00100000:0.0:1787826332.710860:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346256128 00000100:00100000:0.0:1787826332.710904:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23824 00000100:00100000:0.0:1787826332.710912:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256128:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.711588:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256128:12345-0@lo:1000:mdt_io00_002.0 Request processed in 675us (1793us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.711597:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23824 00000100:00100000:3.0:1787826332.711681:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256128:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.711750:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256384:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.711786:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.711820:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b16c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826332.711873:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346256384 00000100:00100000:0.0:1787826332.711879:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346256384 00000100:00100000:0.0:1787826332.712040:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23825 00000100:00100000:0.0:1787826332.712050:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256384:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.712144:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256384:12345-0@lo:1000:mdt_io00_002.0 Request processed in 96us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.712151:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23825 00000100:00100000:2.0:1787826332.712251:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b240 going to sleep for 11 seconds 00000100:00080000:2.0:1787826332.712273:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826332.712434:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256384:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826332.712486:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256512:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826332.712526:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826332.712544:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.712589:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346256512 00000100:00100000:3.0:1787826332.712594:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346256512 00000100:00100000:3.0:1787826332.712672:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23826 00000100:00100000:3.0:1787826332.712681:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256512:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.712790:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256512:12345-0@lo:1000:mdt_io00_002.0 Request processed in 110us (263us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826332.712798:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23826 00000100:00100000:3.0:1787826332.712829:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256512:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.712902:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256640:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.712946:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.712961:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.712973:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346256640 00000100:00100000:3.0:1787826332.712976:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346256640 00000100:00100000:3.0:1787826332.712997:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23827 00000100:00100000:3.0:1787826332.713003:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256640:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.713210:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256640:12345-0@lo:1000:mdt_io00_002.0 Request processed in 206us (263us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826332.713217:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23827 00000100:00100000:3.0:1787826332.713247:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256640:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.713316:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256768:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.713342:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.713358:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.713369:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346256768 00000100:00100000:3.0:1787826332.713373:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346256768 00000100:00100000:3.0:1787826332.713394:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23828 00000100:00100000:3.0:1787826332.713401:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256768:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.713456:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346256768:12345-0@lo:1000:mdt_io00_002.0 Request processed in 55us (112us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826332.713462:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23828 00000100:00100000:3.0:1787826332.713488:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346256768:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826332.713517:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e52:0x0] 00010000:00010000:3.0:1787826332.713586:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.713604:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.713612:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.713637:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.713651:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 9 pid: 24702 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826332.713665:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346256896:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826332.713707:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.718403:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346256896:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826332.718440:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826332.719683:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826332.719753:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332627328 00000100:00100000:0.0:1787826332.719761:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332627328 00010000:00010000:0.0:1787826332.719805:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 10 pid: 24702 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826332.719841:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21988 00000100:00100000:0.0:1787826332.719849:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332627328:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826332.719863:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826332.719870:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 10 pid: 24702 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.719883:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 10 pid: 24702 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.719904:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b410 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016ef expref: 10 pid: 24702 timeout: 8244 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.719929:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826332.719934:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.719943:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.719950:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.719981:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826332.720218:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.720227:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.720284:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.720292:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.720302:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826332.720322:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257024:0@lo:101:mdt_io00_002.0 00000100:00100000:3.0:1787826332.720363:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.720402:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.720444:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346257024 00000100:00100000:1.0:1787826332.720449:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346257024 00000100:00100000:1.0:1787826332.720485:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23829 00000100:00100000:1.0:1787826332.720493:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346257024:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826332.720516:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826332.720533:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b45d lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b456 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.720552:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b45d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b456 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.720564:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b45d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b456 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.720571:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f7967ac1100) 00010000:00010000:1.0:1787826332.720574:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787826332.720576:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826332.720619:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257024:12345-0@lo:101:mdt_io00_002.0 Request processed in 127us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.720625:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23829 00000100:00100000:0.0:1787826332.720700:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332627328:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 850us (1021us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.720708:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21988 00000100:00100000:2.0:1787826332.721183:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257024:0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787826332.721201:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b45d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826332.721211:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b45d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826332.721233:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e53:0x0] 00000100:00100000:2.0:1787826332.721287:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257152:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826332.721323:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826332.721347:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f58b900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826332.721520:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346257152 00000100:00100000:0.0:1787826332.721526:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346257152 00000100:00100000:0.0:1787826332.721568:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23830 00000100:00100000:0.0:1787826332.721577:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257152:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.721663:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257152:12345-0@lo:1000:mdt_io00_002.0 Request processed in 87us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826332.721669:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23830 00000100:00100000:3.0:1787826332.721756:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257152:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826332.721787:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x197:0x0] 00000100:00100000:3.0:1787826332.721820:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257280:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.721850:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.721877:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.721935:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346257280 00000100:00100000:1.0:1787826332.721940:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346257280 00000100:00100000:1.0:1787826332.721974:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23831 00000100:00100000:1.0:1787826332.721982:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257280:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.722056:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257280:12345-0@lo:1000:mdt_io00_002.0 Request processed in 75us (205us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.722063:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23831 00000100:00100000:3.0:1787826332.722316:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257280:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826332.722335:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e52:0x0] 00010000:00010000:3.0:1787826332.722371:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.722384:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.722393:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826332.722435:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257408:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.722457:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.722481:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.722532:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346257408 00000100:00100000:1.0:1787826332.722537:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346257408 00000100:00100000:1.0:1787826332.722565:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23832 00000100:00100000:1.0:1787826332.722571:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257408:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.722616:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257408:12345-0@lo:1000:mdt_io00_002.0 Request processed in 45us (157us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.722622:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23832 00000100:00100000:3.0:1787826332.722679:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257408:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826332.722693:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e52:0x0] 00000100:00100000:3.0:1787826332.722711:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257536:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.722732:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.722751:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.722791:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346257536 00000100:00100000:1.0:1787826332.722795:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346257536 00000100:00100000:1.0:1787826332.722822:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23833 00000100:00100000:1.0:1787826332.722829:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257536:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.722873:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257536:12345-0@lo:1000:mdt_io00_002.0 Request processed in 44us (140us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.722879:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23833 00000100:00100000:3.0:1787826332.723051:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257536:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.723083:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257664:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826332.723139:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826332.723157:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.723341:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346257664 00000100:00100000:1.0:1787826332.723346:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346257664 00000100:00100000:1.0:1787826332.723370:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23834 00000100:00100000:1.0:1787826332.723377:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257664:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.723440:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257664:12345-0@lo:1000:mdt_io00_002.0 Request processed in 63us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.723447:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23834 00000100:00100000:0.0:1787826332.723530:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257664:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826332.723549:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x197:0x0] 00000100:00100000:0.0:1787826332.723575:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc30000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257792:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.723604:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826332.723627:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.723672:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346257792 00000100:00100000:1.0:1787826332.723676:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346257792 00000100:00100000:1.0:1787826332.723705:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23835 00000100:00100000:1.0:1787826332.723712:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257792:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.723762:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257792:12345-0@lo:1000:mdt_io00_002.0 Request processed in 50us (156us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.723768:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23835 00000100:00100000:0.0:1787826332.723828:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc30000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257792:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826332.723847:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e52:0x0] 00000100:00100000:0.0:1787826332.723875:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc30000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257920:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.723899:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826332.723928:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.724221:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346257920 00000100:00100000:1.0:1787826332.724226:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346257920 00000100:00100000:1.0:1787826332.724251:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23836 00000100:00100000:1.0:1787826332.724258:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257920:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.724311:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346257920:12345-0@lo:1000:mdt_io00_002.0 Request processed in 54us (411us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.724317:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23836 00000100:00100000:0.0:1787826332.724399:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc30000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346257920:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.724431:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346258048:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.724454:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826332.724466:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.724517:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346258048 00000100:00100000:1.0:1787826332.724521:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346258048 00000100:00100000:1.0:1787826332.724550:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23837 00000100:00100000:1.0:1787826332.724556:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346258048:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.724620:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346258048:12345-0@lo:1000:mdt_io00_002.0 Request processed in 64us (164us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.724627:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23837 00000100:00100000:0.0:1787826332.724689:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346258048:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.724754:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346258176:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826332.724777:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826332.724799:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826332.724845:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346258176 00000100:00100000:1.0:1787826332.724850:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346258176 00000100:00100000:1.0:1787826332.724880:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23838 00000100:00100000:1.0:1787826332.724887:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346258176:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826332.724950:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346258176:12345-0@lo:1000:mdt_io00_002.0 Request processed in 63us (171us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.724960:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23838 00000100:00100000:0.0:1787826332.725155:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346258176:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826332.725546:0:6542:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1dbd:0x0] is inserted into orphan list, open count = 1 00000040:00080000:0.0:1787826332.725613:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].938, 1240 off883208 00000040:00080000:0.0:1787826332.725667:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].941, 1320 off982592 00000004:00080000:0.0:1787826332.725672:0:6542:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784dedd6c0 with version 1830 00000004:00080000:1.0:1787826332.725792:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1830 rpc_version 1830 00000020:00100000:1.0:1787826332.725815:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826332.725823:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784c6b9500 x1874663346258432/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826332.725841:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6b9500 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346258432:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826332.725881:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826332.725910:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826332.726436:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346258432 00000100:00100000:3.0:1787826332.726443:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346258432 00000100:00100000:3.0:1787826332.726494:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23839 00000100:00100000:3.0:1787826332.726514:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346258432:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826332.726831:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346258432:12345-0@lo:1000:osp_up1-0.0 Request processed in 319us (949us total) trans 4294985427 rc 0/0 00000100:00100000:3.0:1787826332.726840:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23839 00000100:00100000:3.0:1787826332.726899:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6b9500 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346258432:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787826332.727655:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727667:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727675:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727682:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.727686:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727694:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727709:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826332.727712:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826332.727717:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727725:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b44f lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826332.727751:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e53:0x0] 00010000:00010000:0.0:1787826332.727758:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b45d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727766:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b45d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727778:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727785:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727790:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727795:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.727798:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727804:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727813:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a900) 00010000:00010000:0.0:1787826332.727815:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826332.727819:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727830:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b43a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826332.727848:0:6542:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c7680343 reply state 0000000026ff821a transno 8589957424 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727861:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727868:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:0.0:1787826332.727871:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826332.727874:0:6542:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:0.0:1787826332.727877:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x197:0x0] 00010000:00010000:0.0:1787826332.727882:0:6542:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727890:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727898:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727939:0:6542:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826332.727944:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x197:0x0] 00010000:00010000:0.0:1787826332.727949:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b433 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727956:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b433 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727970:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727976:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727982:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727987:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826332.727990:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.727997:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.728004:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:0.0:1787826332.728007:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826332.728010:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826332.728015:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b464 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826332.728310:0:6542:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225557:x1874663332626560:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 46313us (50452us total) trans 8589957424 rc 0/0 00000100:00100000:0.0:1787826332.728319:0:6542:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1232 00010000:00010000:1.0:1787826332.728471:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826332.728479:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.728486:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826332.728490:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826332.728508:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.728542:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.728548:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826332.728577:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.728585:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490eee00/0xe21309aa81f4b409 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b41e expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.728606:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826332.728608:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826332.728610:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826332.728618:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826332.728633:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346258560:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826332.728688:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826332.728737:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346258560 00000100:00100000:1.0:1787826332.728745:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346258560 00000100:00100000:1.0:1787826332.728796:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11134 00000100:00100000:1.0:1787826332.728806:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346258560:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826332.728818:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826332.728836:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b41e lrc: 0/0,0 mode: --/CW res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b409 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.728849:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f7961f9a100) 00010000:00010000:1.0:1787826332.728856:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826332.728884:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346258560:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826332.728890:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11134 00000100:00100000:1.0:1787826332.729295:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346258560:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826332.732516:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826332.732604:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332627456 00000100:00100000:3.0:1787826332.732614:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332627456 00000100:00100000:3.0:1787826332.732679:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68518 00000100:00100000:3.0:1787826332.732690:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225555:x1874663332627456:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826332.732717:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826332.732743:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b46b lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801727 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.732817:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b46b lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801727 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.732830:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b46b lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801727 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.732849:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:3.0:1787826332.732852:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787826332.732856:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826332.733035:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225555:x1874663332627456:12345-192.168.203.56@tcp:101:cat.0 Request processed in 345us (535us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826332.733042:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68518 00000100:00100000:2.0:1787826332.733173:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:17812:1874663346256256:192.168.203.56@tcp:105: 00010000:00010000:2.0:1787826332.733194:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826332.733219:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332627200:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 23077us (23315us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826332.733228:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21987 00000100:00100000:2.0:1787826332.737670:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826332.737749:0:24158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332627584 00000100:00100000:1.0:1787826332.737759:0:24158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332627584 00002000:00100000:1.0:1787826332.737827:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x521:0x0] (0->4095) req@ffff9f784d8bf100 x1874663332627584/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:688/0 lens 488/0 e 0 to 0 dl 1787826343 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826332.737839:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d8bf100 with x1874663332627584 ext(0->4095) 00002000:00010000:1.0:1787826332.737847:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784d8bf100 00000100:00100000:1.0:1787826332.737881:0:24158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27655 00000100:00100000:1.0:1787826332.737892:0:24158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332627584:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:1.0:1787826332.738059:0:24158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332627584:12345-192.168.203.56@tcp:3:cat.0 Request processed in 169us (399us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787826332.738067:0:24158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27655 00002000:00100000:1.0:1787826332.738074:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000401:0x521:0x0] (0->4095) req@ffff9f784d8bf100 x1874663332627584/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:688/0 lens 488/4536 e 0 to 0 dl 1787826343 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826332.738081:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d8bf100 with x1874663332627584 ext(0->4095) 00002000:00010000:1.0:1787826332.738085:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784d8bf100 00000100:00100000:1.0:1787826332.742600:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826332.742686:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332627840 00000100:00100000:3.0:1787826332.742695:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332627840 00000100:00100000:3.0:1787826332.742758:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9361 00000100:00100000:3.0:1787826332.742768:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225555:x1874663332627840:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:3.0:1787826332.742992:0:17601:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1dbd:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1787826332.743068:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].214, 64 off21888 00000004:00080000:3.0:1787826332.743081:0:17601:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8144954975069ns, next at 8145954971916ns, rc = 0 00010000:00010000:3.0:1787826332.743291:0:17601:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.743308:0:17601:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.743319:0:17601:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.743326:0:17601:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.743333:0:17601:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.743339:0:17601:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826332.743361:0:17601:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826332.743558:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225555:x1874663332627840:12345-192.168.203.56@tcp:35:cat.0 Request processed in 790us (966us total) trans 8589957425 rc 0/0 00000100:00100000:3.0:1787826332.743566:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9361 00010000:00010000:1.0:1787826332.743631:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826332.743638:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.743645:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826332.743649:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.743656:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.743668:0:30445:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:1.0:1787826332.743671:0:30445:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826332.743675:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.743681:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b472 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dbd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826332.743694:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826332.743696:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826332.743697:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826332.844412:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826333.235184:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b433 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826333.235207:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b433 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.235255:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b433 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.235262:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826333.235346:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:1.0:1787826333.235425:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecee00/0xe21309aa81f4b42c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b433 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.235449:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b45d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826333.235456:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b45d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.235476:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b45d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.235482:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826333.235516:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826333.235540:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346258816:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826333.235614:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826333.235660:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346258816 00000100:00100000:2.0:1787826333.235667:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346258816 00000100:00100000:2.0:1787826333.235712:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11135 00000100:00100000:2.0:1787826333.235721:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f4850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346258816:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826333.235733:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826333.235751:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4b45d lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b456 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826333.235767:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f7967ac1100) 00010000:00010000:2.0:1787826333.235776:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826333.235810:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f4850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346258816:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 90us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826333.235817:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11135 00000100:00100000:2.0:1787826333.236557:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346258816:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826333.238659:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346258688:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826333.238733:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826333.238845:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346258688 00000100:00100000:3.0:1787826333.238853:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346258688 00000100:00100000:3.0:1787826333.238912:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11136 00000100:00100000:3.0:1787826333.238921:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346258688:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826333.238935:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826333.239029:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4b433 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x197:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b42c expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.239048:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x4ec94fd6 (ffff9f79785cc800) 00010000:00010000:3.0:1787826333.239059:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826333.239097:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346258688:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 176us (363us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826333.239119:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11136 00000100:00100000:3.0:1787826333.239165:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346258688:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826333.239656:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69e00/0xe21309aa81f4b456 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b45d expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826333.239683:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957425 is committed 00000004:00080000:1.0:1787826333.239690:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 219 committed 00000100:00100000:2.0:1787826333.239902:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826333.239915:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346258944:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787826333.239960:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826333.240294:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346258944 00000100:00100000:3.0:1787826333.240301:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346258944 00000001:00080000:1.0:1787826333.240578:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957424 is committed 00000020:00080000:1.0:1787826333.240595:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979084 batchid updated 00000001:00080000:1.0:1787826333.240604:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957422 is committed 00010000:00010000:1.0:1787826333.240642:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.240652:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.240659:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.240665:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826333.240672:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.240679:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.240694:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:1.0:1787826333.240697:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826333.240701:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.240706:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4b2e3 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787826333.240727:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979084 00000004:00080000:1.0:1787826333.240790:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784c4f4780 with version 1831 00000020:00080000:1.0:1787826333.240801:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979084 cancel update log [0x240000404:0x2:0x0].940: rc = 0 00000100:00100000:3.0:1787826333.241575:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30851 00000100:00100000:3.0:1787826333.241588:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346258944:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1787826333.241618:0:35097:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1313 count 1 00010000:00010000:3.0:1787826333.241643:0:35097:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.241665:0:35097:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.241673:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=1s v=(1 0) 00010000:00010000:3.0:1787826333.241677:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826333.241682:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.241713:0:35097:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 5 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.241732:0:35097:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 5 pid: 35096 timeout: 8245 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826333.241747:0:35097:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:35097:1874663346259072:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826333.241845:0:35097:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1900 going to sleep for 11 seconds 00000004:00080000:3.0:1787826333.241861:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1831 rpc_version 1831 00000020:00100000:3.0:1787826333.241882:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826333.241891:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784daaad80 x1874663346259328/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826333.241908:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaad80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346259328:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826333.242200:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826333.242230:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826333.242244:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346259328 00000100:00100000:3.0:1787826333.242250:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346259328 00000100:00100000:3.0:1787826333.242283:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23840 00000100:00100000:3.0:1787826333.242291:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346259328:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826333.242579:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346259328:12345-0@lo:1000:osp_up1-0.0 Request processed in 288us (379us total) trans 4294985428 rc 0/0 00000100:00100000:3.0:1787826333.242587:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23840 00000100:00100000:3.0:1787826333.242647:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaad80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346259328:0@lo:1000:osp_up1-0.0 00000100:00080000:2.0:1787826333.242907:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787826333.242927:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826333.242941:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826333.243022:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826333.243028:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826333.243034:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826333.243041:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826333.243056:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f784b4a5c00) 00010000:00010000:2.0:1787826333.243059:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826333.243064:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826333.243070:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b448 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787826333.244437:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979084 cancel update log [0x200000bd0:0x2:0x0].937: rc = 0 00000020:00080000:1.0:1787826333.244455:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979084 00000100:00100000:3.0:1787826333.248693:0:35097:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:35097:1874663346259072:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826333.248731:0:35097:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826333.255277:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826333.255821:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332627968 00000100:00100000:1.0:1787826333.255832:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332627968 00010000:00010000:1.0:1787826333.255895:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 8245 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826333.255943:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21989 00000100:00100000:1.0:1787826333.255954:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332627968:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826333.255969:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826333.255976:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 8245 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.255990:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 8245 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.256014:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004857cdca/0xe21309aa81f4b441 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801720 expref: 6 pid: 35096 timeout: 8245 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.256032:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826333.256037:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826333.256043:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.256058:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.256093:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826333.256143:0:35097:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.256154:0:35097:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.256162:0:35097:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.256169:0:35097:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826333.256190:0:35097:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826333.256193:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332627968:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 241us (926us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826333.256200:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21989 00010000:00010000:1.0:1787826333.256343:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826333.256350:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.256358:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826333.256362:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.256372:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826333.256372:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346258944:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14787us (16410us total) trans 4294976163 rc 0/0 00000100:00100000:3.0:1787826333.256380:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30851 00010000:00010000:1.0:1787826333.256383:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.256389:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ae043674/0xe21309aa81f4b479 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826333.256407:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826333.256410:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826333.256412:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1787826333.256432:0:3655:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784d8be300/2, rc 0, transno 4294976163 00000100:00100000:3.0:1787826333.256438:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346258944:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1787826333.868356:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826334.085949:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.092629:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332628096 00000100:00100000:2.0:1787826334.092644:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332628096 00000100:00100000:2.0:1787826334.092787:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68519 00000100:00100000:2.0:1787826334.092800:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225593:x1874663332628096:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826334.092821:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.092847:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b480 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080172e expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.092925:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b487 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.092939:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b487 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.092951:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b487 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.093061:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e52:0x0]: returning size 4096 00000004:00200000:2.0:1787826334.093065:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787826334.093085:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b487 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.094463:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b480 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080172e expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.094511:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b487 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080172e expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.094520:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:2.0:1787826334.094523:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826334.094526:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826334.094612:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225593:x1874663332628096:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1814us (8670us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.094622:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68519 00000100:00080000:1.0:1787826334.101420:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826334.102961:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.103361:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332628224 00000100:00100000:2.0:1787826334.103371:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332628224 00000100:00100000:2.0:1787826334.103438:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68520 00000100:00100000:2.0:1787826334.103449:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225593:x1874663332628224:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826334.103470:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.103506:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b48e lrc: 2/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801735 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.103577:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.103591:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.103601:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.103650:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x197:0x0]: returning size 4096 00000004:00200000:2.0:1787826334.103656:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826334.103696:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b48e lrc: 0/0,0 mode: --/CR res: [0x240000400:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.103727:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.103735:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f7961f9a800) 00010000:00010000:2.0:1787826334.103737:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826334.103740:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826334.107474:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225593:x1874663332628224:12345-192.168.203.56@tcp:101:rm.0 Request processed in 4015us (4520us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.107487:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68520 00000100:00100000:0.0:1787826334.112910:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.112983:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332628352 00000100:00100000:2.0:1787826334.112993:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332628352 00000100:00100000:2.0:1787826334.113136:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68521 00000100:00100000:2.0:1787826334.113148:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225593:x1874663332628352:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826334.113181:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.113206:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b49c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080173c expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.113755:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.113781:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.113792:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.113838:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x195:0x0]: returning size 4096 00000004:00200000:2.0:1787826334.113844:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826334.113884:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b49c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.113901:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.113909:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a400) 00010000:00010000:2.0:1787826334.113913:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826334.113917:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826334.113981:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225593:x1874663332628352:12345-192.168.203.56@tcp:101:rm.0 Request processed in 836us (1080us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.113988:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68521 00000100:00080000:3.0:1787826334.116208:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826334.128958:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.133187:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332628608 00000100:00100000:0.0:1787826334.133200:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332628608 00000100:00100000:2.0:1787826334.133667:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68522 00000100:00100000:2.0:1787826334.133681:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225593:x1874663332628608:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826334.133701:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.133734:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b4aa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801743 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.133885:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e52:0x0]: returning size 4096 00010000:00010000:2.0:1787826334.133989:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b4aa lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801743 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.134003:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7961f9a500) 00010000:00010000:2.0:1787826334.134006:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826334.134203:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b4aa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801743 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.134225:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826334.134355:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225593:x1874663332628608:12345-192.168.203.56@tcp:101:rm.0 Request processed in 678us (5406us total) trans 4294985429 rc 301/301 00000100:00100000:2.0:1787826334.134362:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68522 00000100:00100000:3.0:1787826334.141830:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826334.141954:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332628736 00000100:00100000:1.0:1787826334.141964:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332628736 00000100:00100000:1.0:1787826334.142028:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9362 00000100:00100000:1.0:1787826334.142039:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943033450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225593:x1874663332628736:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826334.158937:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943033450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225593:x1874663332628736:12345-192.168.203.56@tcp:37:rm.0 Request processed in 16887us (17104us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.158952:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9362 00000100:00100000:1.0:1787826334.178019:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.179292:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332628864 00000100:00100000:2.0:1787826334.179304:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332628864 00000100:00100000:2.0:1787826334.179365:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9363 00000100:00100000:2.0:1787826334.179376:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225593:x1874663332628864:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826334.200962:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225593:x1874663332628864:12345-192.168.203.56@tcp:37:rm.0 Request processed in 21579us (22941us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.200977:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9363 00000100:00100000:0.0:1787826334.214942:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.216646:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332629120 00000100:00100000:0.0:1787826334.216658:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332629120 00000100:00100000:0.0:1787826334.216723:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9364 00000100:00100000:0.0:1787826334.216734:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225593:x1874663332629120:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:0.0:1787826334.217017:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1211, 344 off1103616 00000040:00080000:0.0:1787826334.217068:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1530, 424 off1547176 00000004:00080000:0.0:1787826334.217074:0:6531:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7849247780 with version 6852 00000004:00080000:2.0:1787826334.219514:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6852 rpc_version 6852 00000020:00100000:2.0:1787826334.219556:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826334.219568:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784f541500 x1874663346259584/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826334.219594:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346259584:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826334.219665:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.219701:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f37000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826334.219728:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346259584 00000100:00100000:2.0:1787826334.219735:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346259584 00000100:00100000:2.0:1787826334.219795:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23841 00000100:00100000:2.0:1787826334.219805:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346259584:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826334.220000:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346259584:12345-0@lo:1000:osp_up0-1.0 Request processed in 196us (334us total) trans 8589957426 rc 0/0 00000100:00100000:2.0:1787826334.220019:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23841 00000100:00100000:2.0:1787826334.220062:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957422 gen 2 00000004:00080000:2.0:1787826334.220067:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957419 committed transno 8589957422 00000004:00080000:2.0:1787826334.220085:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957420 committed transno 8589957422 00000004:00080000:2.0:1787826334.220098:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957421 committed transno 8589957422 00000004:00080000:2.0:1787826334.220138:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957422 committed transno 8589957422 00000100:00100000:2.0:1787826334.220152:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f784f541500 x1874663346259584/t8589957426(8589957426) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826350 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826334.220179:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346259584:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826334.220463:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225593:x1874663332629120:12345-192.168.203.56@tcp:35:rm.0 Request processed in 3728us (5526us total) trans 4294985430 rc 0/0 00000100:00100000:2.0:1787826334.220476:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9364 00000100:00100000:2.0:1787826334.232590:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.232635:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332629376 00000100:00100000:2.0:1787826334.232644:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332629376 00000100:00100000:2.0:1787826334.232698:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68523 00000100:00100000:2.0:1787826334.232708:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225593:x1874663332629376:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826334.232732:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.234119:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4b4b8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080174a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.234396:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b4bf lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.234410:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b4bf lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.234422:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b4bf lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.234436:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e52:0x0]: returning size 4096 00010000:00010000:2.0:1787826334.234555:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efa00/0xe21309aa81f4b4b8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080174a expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.234582:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b4bf lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080174a expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.234592:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:2.0:1787826334.234595:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1787826334.234599:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826334.234666:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225593:x1874663332629376:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1958us (2085us total) trans 4294985431 rc 0/0 00000100:00100000:2.0:1787826334.234675:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68523 00000100:00100000:2.0:1787826334.246863:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826334.246959:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332629504 00000100:00100000:3.0:1787826334.246970:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332629504 00000100:00100000:3.0:1787826334.247140:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68524 00000100:00100000:3.0:1787826334.247153:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225593:x1874663332629504:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:3.0:1787826334.247214:0:6528:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345561600 00000100:00100000:3.0:1787826334.248658:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225593:x1874663332629504:12345-192.168.203.56@tcp:41:rm.0 Request processed in 221us (519us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.248680:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68524 00000100:00100000:2.0:1787826334.254166:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826334.255064:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332629632 00000100:00100000:3.0:1787826334.255075:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332629632 00000100:00100000:3.0:1787826334.255175:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68525 00000100:00100000:3.0:1787826334.255186:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225593:x1874663332629632:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826334.255208:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826334.255231:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b4cd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801751 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256167:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256183:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256194:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256210:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 3/1,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256216:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256225:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256297:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b4e2 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256305:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b4e2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256311:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b4e2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826334.256333:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b4e2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256347:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256353:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256359:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256365:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256370:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826334.256387:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256394:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256405:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a400) 00010000:00010000:3.0:1787826334.256408:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.256412:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256417:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b4d4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256431:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256437:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256441:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256445:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826334.256448:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256453:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256458:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826334.256460:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.256462:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 2/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256467:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b4db lrc: 0/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256513:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b4cd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801751 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256527:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b4e2 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801751 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.256533:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f79785cc600) 00010000:00010000:3.0:1787826334.256535:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826334.256537:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826334.256607:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225593:x1874663332629632:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1422us (2451us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.256614:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68525 00000100:00100000:1.0:1787826334.260849:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.261000:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332629760 00000100:00100000:2.0:1787826334.261021:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332629760 00000100:00100000:2.0:1787826334.261090:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68526 00000100:00100000:2.0:1787826334.261101:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225593:x1874663332629760:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826334.261167:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.261192:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4b4e9 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801758 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.261254:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b4f0 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.261266:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b4f0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.261277:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b4f0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.261332:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e53:0x0]: returning size 6 00000004:00200000:2.0:1787826334.261337:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:2.0:1787826334.261373:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4b4e9 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801758 expref: 14 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.261391:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b4f0 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801758 expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.261399:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f7961f9a500) 00010000:00010000:2.0:1787826334.261402:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826334.261405:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:2.0:1787826334.261466:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225593:x1874663332629760:12345-192.168.203.56@tcp:101:rm.0 Request processed in 366us (627us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.261472:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68526 00000100:00100000:3.0:1787826334.269904:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826334.269954:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826334.270077:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332629888 00000100:00100000:1.0:1787826334.270087:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332629888 00000100:00100000:1.0:1787826334.270185:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11137 00000100:00100000:1.0:1787826334.270196:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2395:x1874663332629888:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:1.0:1787826334.270211:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826334.270238:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b4e2 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801751 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.270267:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f79785cc600) 00010000:00010000:1.0:1787826334.270278:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826334.270333:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2395:x1874663332629888:12345-192.168.203.56@tcp:103:rm.0 Request processed in 139us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.270340:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11137 00000100:00100000:1.0:1787826334.270364:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332630144 00000100:00100000:1.0:1787826334.270368:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332630144 00000100:00100000:1.0:1787826334.270410:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68527 00000100:00100000:1.0:1787826334.270417:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225593:x1874663332630144:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826334.270463:0:35093:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826334.270480:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b4f0 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e53:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801758 expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.270504:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f7961f9a500) 00010000:00010000:1.0:1787826334.270513:0:35093:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826334.270561:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.270571:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.270580:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.270603:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346259712:0@lo:101:mdt00_010.0 00000100:00100000:1.0:1787826334.270647:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.270677:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.270698:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346259712 00000100:00100000:1.0:1787826334.270701:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346259712 00000100:00100000:1.0:1787826334.270728:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23842 00000100:00100000:1.0:1787826334.270735:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35093:x1874663346259712:12345-0@lo:101:mdt00_010.0 00010000:00010000:1.0:1787826334.270749:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826334.270774:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 2/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b4f7 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.270795:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.270810:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 4/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b4f7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.270835:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.270847:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 9 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.270857:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346259840:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826334.270900:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.274608:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346259840:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826334.274651:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 3/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b4f7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.274663:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a400) 00010000:00010000:3.0:1787826334.274666:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.274670:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 3/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b4f7 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.274679:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:3.0:1787826334.274746:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346259712:12345-0@lo:101:mdt00_010.0 Request processed in 4011us (4097us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.274765:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23842 00000100:00100000:1.0:1787826334.274924:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346259712:0@lo:101:mdt00_010.0 00010000:00010000:1.0:1787826334.274942:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.274951:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.279909:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.279996:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332630272 00000100:00100000:0.0:1787826334.280015:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332630272 00010000:00010000:0.0:1787826334.280063:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 10 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826334.280136:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21990 00000100:00100000:0.0:1787826334.280146:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332630272:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826334.280162:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826334.280169:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 10 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.280182:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 10 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.280205:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b4a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x195:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080173c expref: 10 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.280221:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a400) 00010000:00010000:0.0:1787826334.280229:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 2/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b4f7 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.280240:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b4f7 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.280248:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b4f7 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.280426:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b4f7 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826334.280442:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346259968:0@lo:105: 00000100:00100000:0.0:1787826334.280506:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826334.280544:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.280677:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346259968 00000100:00100000:3.0:1787826334.280682:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346259968 00000100:00100000:3.0:1787826334.280729:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7885 00000100:00100000:3.0:1787826334.280736:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346259968:12345-0@lo:105: 00010000:00010000:3.0:1787826334.280746:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.280756:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.280765:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.280773:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.280779:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.280787:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.280805:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf800) 00000100:00100000:3.0:1787826334.280846:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346259968:12345-0@lo:105: Request processed in 111us (339us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.280856:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7885 00000100:00100000:3.0:1787826334.280893:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346259968:0@lo:105: 00010000:00010000:3.0:1787826334.280911:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826334.280952:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332630272:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 807us (1053us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.280958:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21990 00010000:00010000:2.0:1787826334.281094:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.281126:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826334.281151:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x195:0x0] 00010000:00010000:2.0:1787826334.281187:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.281194:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.281203:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826334.281220:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260096:0@lo:101:mdt00_010.0 00000100:00100000:2.0:1787826334.281264:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.281290:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ae40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.281870:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346260096 00000100:00100000:3.0:1787826334.281877:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346260096 00000100:00100000:3.0:1787826334.281926:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23843 00000100:00100000:3.0:1787826334.281934:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8b850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346260096:12345-0@lo:101:mdt00_010.0 00010000:00010000:3.0:1787826334.281950:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826334.281967:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b50c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b505 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.281988:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b50c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b505 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.282001:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b50c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b505 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.282534:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826334.282537:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.282540:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826334.282599:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8b850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346260096:12345-0@lo:101:mdt00_010.0 Request processed in 665us (1333us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.282606:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23843 00000100:00100000:2.0:1787826334.282892:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260096:0@lo:101:mdt00_010.0 00010000:00010000:2.0:1787826334.282910:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b50c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.282921:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b50c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826334.282941:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x195:0x0] 00000100:00100000:2.0:1787826334.282983:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260224:0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826334.285377:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826334.285776:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346260224 00000100:00100000:0.0:1787826334.285786:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346260224 00000100:00100000:0.0:1787826334.285841:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23844 00000100:00100000:0.0:1787826334.285850:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346260224:12345-0@lo:1000:mdt00_010.0 00000100:00100000:0.0:1787826334.285957:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346260224:12345-0@lo:1000:mdt00_010.0 Request processed in 109us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.285965:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23844 00000100:00100000:3.0:1787826334.287387:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.287433:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260224:0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826334.287539:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9500 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260352:0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826334.287599:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.287622:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.287649:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346260352 00000100:00100000:3.0:1787826334.287654:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346260352 00000100:00100000:3.0:1787826334.287697:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23845 00000100:00100000:3.0:1787826334.287705:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346260352:12345-0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826334.287786:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346260352:12345-0@lo:1000:mdt00_010.0 Request processed in 82us (187us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.287794:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23845 00000100:00100000:3.0:1787826334.287823:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9500 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260352:0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826334.287864:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9500 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260480:0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826334.287886:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.287902:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.287915:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346260480 00000100:00100000:3.0:1787826334.287918:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346260480 00000100:00100000:3.0:1787826334.287940:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23846 00000100:00100000:3.0:1787826334.287946:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346260480:12345-0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826334.288013:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346260480:12345-0@lo:1000:mdt00_010.0 Request processed in 67us (126us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826334.288019:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23846 00000100:00100000:3.0:1787826334.288043:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9500 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260480:0@lo:1000:mdt00_010.0 00010000:00010000:3.0:1787826334.288095:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.288131:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.288141:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.288154:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9500 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260608:0@lo:101:mdt00_010.0 00000100:00100000:3.0:1787826334.288178:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.288191:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.288202:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346260608 00000100:00100000:3.0:1787826334.288205:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346260608 00000100:00100000:3.0:1787826334.288224:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23847 00000100:00100000:3.0:1787826334.288229:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346260608:12345-0@lo:101:mdt00_010.0 00010000:00010000:3.0:1787826334.288238:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826334.289542:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b51a lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b513 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.289580:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b51a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b513 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.289594:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b51a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b513 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.289602:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826334.289605:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826334.289609:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826334.289674:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35093:x1874663346260608:12345-0@lo:101:mdt00_010.0 Request processed in 1444us (1494us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.289684:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23847 00000100:00100000:3.0:1787826334.289739:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9500 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346260608:0@lo:101:mdt00_010.0 00010000:00010000:3.0:1787826334.289757:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.289767:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.289801:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.289809:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.289815:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787826334.290176:0:35093:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].228, 64 off22784 00000004:00080000:3.0:1787826334.290188:0:35093:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8146502081667ns, next at 8147502079087ns, rc = 0 00000040:00080000:3.0:1787826334.290217:0:35093:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1212, 384 off1104000 00000040:00080000:3.0:1787826334.290274:0:35093:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1531, 464 off1547640 00000004:00080000:3.0:1787826334.290280:0:35093:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81000 with version 6853 00000004:00080000:2.0:1787826334.292291:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6853 rpc_version 6853 00000020:00100000:2.0:1787826334.292331:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826334.292342:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784c6bb480 x1874663346260864/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826334.292367:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346260864:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826334.292433:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.292472:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784deddc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826334.296618:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346260864 00000100:00100000:0.0:1787826334.296633:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346260864 00000100:00100000:0.0:1787826334.296694:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23848 00000100:00100000:0.0:1787826334.296705:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346260864:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826334.297679:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346260864:12345-0@lo:1000:osp_up0-1.0 Request processed in 972us (5240us total) trans 8589957427 rc 0/0 00000100:00100000:0.0:1787826334.297692:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23848 00000100:00100000:2.0:1787826334.297825:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346260864:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787826334.297912:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.297931:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.297946:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.297956:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.297962:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.297968:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.297988:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826334.298019:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298026:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298032:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298038:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826334.298043:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298050:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298062:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:3.0:1787826334.298065:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.298069:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298078:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b521 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298091:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298099:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298144:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298161:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787826334.298177:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x195:0x0] 00010000:00010000:3.0:1787826334.298185:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298193:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298199:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298214:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787826334.298218:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x195:0x0] 00010000:00010000:3.0:1787826334.298223:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b50c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.298235:0:35093:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b50c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.298316:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225593:x1874663332630144:12345-192.168.203.56@tcp:36:rm.0 Request processed in 27896us (28358us total) trans 4294985432 rc 0/0 00000100:00100000:3.0:1787826334.298323:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68527 00010000:00010000:0.0:1787826334.298961:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787826334.298970:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.298978:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826334.298981:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.298988:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.298999:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:0.0:1787826334.299002:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826334.299014:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299021:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b528 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e53:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299038:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826334.299042:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299048:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826334.299052:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826334.299058:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299097:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299125:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826334.299150:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299157:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b513 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b51a expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299169:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826334.299172:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299178:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826334.299182:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826334.299189:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299205:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299211:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826334.299227:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299237:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b4f7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b4fe expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299258:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826334.299260:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826334.299261:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826334.299270:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826334.299287:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843092a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346260992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826334.299345:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826334.299386:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346261120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826334.299410:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826334.299449:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346260992 00000100:00100000:0.0:1787826334.299457:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346260992 00000100:00100000:0.0:1787826334.299511:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346261120 00000100:00100000:0.0:1787826334.299515:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346261120 00000100:00100000:0.0:1787826334.299538:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11138 00000100:00100000:0.0:1787826334.299547:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346260992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826334.299559:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826334.299578:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b51a lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e53:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b513 expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299589:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e53:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:0.0:1787826334.299596:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826334.299629:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346260992:12345-0@lo:103:ldlm_bl.0 Request processed in 83us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.299636:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11138 00000100:00100000:0.0:1787826334.299651:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11139 00000100:00100000:0.0:1787826334.299657:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346261120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826334.299664:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826334.299674:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b4fe lrc: 0/0,0 mode: --/CW res: [0x200000401:0x195:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b4f7 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.299684:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f7961f9a400) 00010000:00010000:0.0:1787826334.299693:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826334.299718:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346261120:12345-0@lo:103:ldlm_bl.0 Request processed in 61us (307us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.299723:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11139 00000100:00100000:0.0:1787826334.299765:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346261120:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826334.299790:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843092a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346260992:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826334.300054:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.300805:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826334.305866:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826334.305996:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.306068:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332630656 00000100:00100000:0.0:1787826334.306078:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332630656 00000100:00100000:0.0:1787826334.306164:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9365 00000100:00100000:0.0:1787826334.306174:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225593:x1874663332630656:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:0.0:1787826334.306592:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:225593:x1874663332630656:12345-192.168.203.56@tcp:35:rm.0 Request processed in 417us (593us total) trans 4294985433 rc 0/0 00000100:00100000:0.0:1787826334.306600:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9365 00000100:00100000:2.0:1787826334.306862:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332630400 00000100:00100000:2.0:1787826334.306867:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332630400 00000100:00100000:2.0:1787826334.306916:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11140 00000100:00100000:2.0:1787826334.306924:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f5050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2395:x1874663332630400:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826334.306935:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826334.306961:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4b353 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016d3 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.306975:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:2.0:1787826334.306979:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826334.307022:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f5050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2395:x1874663332630400:12345-192.168.203.56@tcp:103:rm.0 Request processed in 100us (1166us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.307029:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11140 00000100:00100000:3.0:1787826334.311279:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.311395:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332630912 00000100:00100000:0.0:1787826334.311405:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332630912 00000100:00100000:0.0:1787826334.311468:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68528 00000100:00100000:0.0:1787826334.311477:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:225593:x1874663332630912:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826334.311553:0:35092:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826334.311583:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b4bf lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e52:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080174a expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.311605:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4b487 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080172e expref: 13 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.311619:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b361 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016da expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.311630:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:0.0:1787826334.311634:0:35092:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826334.311680:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.311689:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.311699:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826334.311723:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346261248:0@lo:101:mdt00_009.0 00000100:00100000:0.0:1787826334.311787:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826334.311821:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.312194:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346261248 00000100:00100000:3.0:1787826334.312200:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346261248 00000100:00100000:3.0:1787826334.312243:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23849 00000100:00100000:3.0:1787826334.312362:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35092:x1874663346261248:12345-0@lo:101:mdt00_009.0 00010000:00010000:3.0:1787826334.312426:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826334.312443:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b52f expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.312467:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.312475:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.312483:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.312525:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.312538:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 7 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.312550:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346261376:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826334.312635:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.312644:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 7 pid: 35093 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.312652:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346261504:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826334.312676:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826334.317225:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346261376:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826334.317244:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346261504:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826334.317295:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.317308:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826334.317311:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826334.317315:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.317326:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:2.0:1787826334.317409:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35092:x1874663346261248:12345-0@lo:101:mdt00_009.0 Request processed in 5051us (5621us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.317417:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23849 00000100:00100000:0.0:1787826334.317515:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346261248:0@lo:101:mdt00_009.0 00010000:00010000:0.0:1787826334.317535:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.317544:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826334.318916:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.319005:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.319484:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332631040 00000100:00100000:0.0:1787826334.319501:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332631040 00010000:00010000:0.0:1787826334.319599:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 8 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826334.319634:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826334.319668:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332631168 00000100:00100000:3.0:1787826334.319672:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332631168 00010000:00010000:3.0:1787826334.319693:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 8 pid: 35093 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.319726:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21991 00000100:00100000:3.0:1787826334.319735:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332631040:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826334.319748:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826334.319763:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 8 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.319781:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 8 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.319804:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b3a0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016f6 expref: 8 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.319823:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826334.319831:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.319840:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826334.319900:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332631040:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 166us (990us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.319906:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21991 00000100:00100000:3.0:1787826334.319926:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21992 00000100:00100000:3.0:1787826334.319933:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332631168:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826334.319940:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826334.319945:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 8 pid: 35093 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.319955:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 8 pid: 35093 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.319968:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4b337 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016cc expref: 8 pid: 35093 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.319982:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826334.319987:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.319996:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.320005:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.321465:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.321482:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346261632:0@lo:105: 00000100:00100000:3.0:1787826334.321549:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.321594:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.321742:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346261632 00000100:00100000:1.0:1787826334.321750:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346261632 00000100:00100000:1.0:1787826334.321811:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7886 00000100:00100000:1.0:1787826334.321820:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346261632:12345-0@lo:105: 00010000:00010000:1.0:1787826334.321832:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.321841:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.321849:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.321856:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.321863:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.321871:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.321894:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42e00) 00000100:00100000:1.0:1787826334.321938:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346261632:12345-0@lo:105: Request processed in 119us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.321944:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7886 00010000:00010000:1.0:1787826334.321968:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.321976:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826334.322002:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826334.322051:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.322059:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.322067:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.322082:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346261760:0@lo:101:mdt00_009.0 00000100:00100000:1.0:1787826334.322136:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.322155:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.322169:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346261760 00000100:00100000:1.0:1787826334.322173:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346261760 00000100:00100000:1.0:1787826334.322201:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23850 00000100:00100000:1.0:1787826334.322207:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:35092:x1874663346261760:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826334.322221:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826334.322236:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b544 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b53d expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.322256:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b544 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b53d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.322267:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b544 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b53d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.322274:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794385ad00) 00010000:00010000:1.0:1787826334.322277:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826334.322280:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826334.322310:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35092:x1874663346261760:12345-0@lo:101:mdt00_009.0 Request processed in 103us (173us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.322316:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23850 00000100:00100000:1.0:1787826334.322349:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346261632:0@lo:105: 00010000:00010000:1.0:1787826334.322361:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826334.322414:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332631168:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2480us (3407us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.322421:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21992 00000100:00100000:1.0:1787826334.322539:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346261760:0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826334.322551:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.322560:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826334.322572:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787826334.322603:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346261888:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826334.322631:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.322647:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.322661:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346261888 00000100:00100000:1.0:1787826334.322664:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346261888 00000100:00100000:1.0:1787826334.322685:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23851 00000100:00100000:1.0:1787826334.322691:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35092:x1874663346261888:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826334.322747:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35092:x1874663346261888:12345-0@lo:1000:mdt00_009.0 Request processed in 55us (115us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.322754:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23851 00000100:00100000:1.0:1787826334.322782:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346261888:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826334.322823:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346262016:0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826334.322844:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.322857:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.322867:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346262016 00000100:00100000:1.0:1787826334.322870:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346262016 00000100:00100000:1.0:1787826334.322889:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23852 00000100:00100000:1.0:1787826334.322894:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35092:x1874663346262016:12345-0@lo:1000:mdt00_009.0 00000100:00100000:1.0:1787826334.322980:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35092:x1874663346262016:12345-0@lo:1000:mdt00_009.0 Request processed in 84us (134us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826334.322987:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23852 00000100:00100000:1.0:1787826334.323012:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346262016:0@lo:1000:mdt00_009.0 00010000:00010000:1.0:1787826334.323054:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.323061:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.323068:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.323078:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346262144:0@lo:101:mdt00_009.0 00000100:00100000:1.0:1787826334.323101:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.323134:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b1e8900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.323147:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346262144 00000100:00100000:1.0:1787826334.323150:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346262144 00000100:00100000:1.0:1787826334.323171:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23853 00000100:00100000:1.0:1787826334.323176:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35092:x1874663346262144:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826334.323183:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826334.323196:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b54b expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.323215:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 6 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.323225:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b54b expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.323247:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 6 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.323260:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 6 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.323271:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346262272:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826334.323316:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f797cd23f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.327215:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.327412:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346262272:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826334.327454:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b54b expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.327468:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:2.0:1787826334.327471:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826334.327475:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b54b expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.327486:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:2.0:1787826334.327559:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:35092:x1874663346262144:12345-0@lo:101:mdt00_009.0 Request processed in 4382us (4456us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.327568:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23853 00000100:00100000:1.0:1787826334.327664:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346262144:0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826334.327682:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.327692:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.327891:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332631296 00000100:00100000:3.0:1787826334.327899:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332631296 00010000:00010000:3.0:1787826334.327935:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 7 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.327968:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21993 00000100:00100000:3.0:1787826334.327978:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d24000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332631296:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826334.327993:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826334.328000:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 7 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.328026:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 7 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.328046:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b3bc lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608016fd expref: 7 pid: 6527 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.328060:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:3.0:1787826334.328064:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b54b expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.328073:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b54b expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.328080:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b54b expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.328126:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b54b expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.328140:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346262400:0@lo:105: 00000100:00100000:3.0:1787826334.328179:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.328210:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.328486:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346262400 00000100:00100000:1.0:1787826334.328500:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346262400 00000100:00100000:1.0:1787826334.328538:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7887 00000100:00100000:1.0:1787826334.328546:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346262400:12345-0@lo:105: 00010000:00010000:1.0:1787826334.328556:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.328565:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.328574:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.328581:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.328588:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.328595:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.328621:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece800) 00000100:00100000:1.0:1787826334.328659:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346262400:12345-0@lo:105: Request processed in 114us (479us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.328666:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7887 00000100:00100000:3.0:1787826334.328822:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346262400:0@lo:105: 00010000:00010000:3.0:1787826334.328838:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826334.329071:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d24000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332631296:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1029us (1800us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.329084:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21993 00010000:00010000:3.0:1787826334.329212:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329222:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329326:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329338:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329348:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329367:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329445:0:35092:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329453:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329481:0:35092:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.329503:0:35092:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 11 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.329520:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001_UUID:35092:1874663346262528:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826334.329565:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.338506:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.338590:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332631424 00000100:00100000:0.0:1787826334.338601:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332631424 00010000:00010000:0.0:1787826334.338649:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 12 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826334.338690:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21994 00000100:00100000:0.0:1787826334.338703:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332631424:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826334.338716:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826334.338723:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 12 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.338735:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 12 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.338766:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f7961f9a800) 00010000:00010000:0.0:1787826334.338772:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.338781:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.338787:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.338808:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826334.338876:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332631424:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 177us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.338883:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21994 00010000:00010000:3.0:1787826334.341049:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b495 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801735 expref: 11 pid: 24703 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.341075:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001_UUID:35092:1874663346262528:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826334.341102:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.341134:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.341140:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.341194:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.341203:0:35092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.341214:0:35092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.341235:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346262656:0@lo:101:mdt00_009.0 00000100:00100000:3.0:1787826334.341524:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.341566:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.341632:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346262656 00000100:00100000:1.0:1787826334.341640:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346262656 00000100:00100000:1.0:1787826334.341698:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23854 00000100:00100000:1.0:1787826334.341709:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:35092:x1874663346262656:12345-0@lo:101:mdt00_009.0 00010000:00010000:1.0:1787826334.341723:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826334.341740:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b56e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x195:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b567 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.341772:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b56e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b567 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.341785:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b56e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b567 expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.341793:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f794385a100) 00010000:00010000:1.0:1787826334.341796:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826334.341799:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826334.341844:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35092:x1874663346262656:12345-0@lo:101:mdt00_009.0 Request processed in 136us (320us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.341850:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23854 00000100:00100000:3.0:1787826334.342026:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854624e00 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346262656:0@lo:101:mdt00_009.0 00010000:00010000:3.0:1787826334.342041:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.342052:0:35092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.343598:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346262784:0@lo:403:mdt00_009.0 00000100:00100000:3.0:1787826334.343672:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.343750:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826334.344602:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346262784 00000100:00100000:2.0:1787826334.344612:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346262784 00000100:00100000:2.0:1787826334.344739:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23855 00000100:00100000:2.0:1787826334.344749:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:35092:x1874663346262784:12345-0@lo:403:mdt00_009.0 00000100:00100000:2.0:1787826334.348737:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:35092:x1874663346262784:12345-0@lo:403:mdt00_009.0 Request processed in 3980us (5054us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.348963:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23855 00000100:00100000:3.0:1787826334.350300:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346262784:0@lo:403:mdt00_009.0 00000040:00080000:2.0:1787826334.352525:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1213, 952 off1104952 00000040:00080000:2.0:1787826334.352712:0:35092:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1532, 1032 off1548672 00000004:00080000:2.0:1787826334.352718:0:35092:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7844e81a80 with version 6854 00000004:00080000:3.0:1787826334.355875:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6854 rpc_version 6854 00000020:00100000:0.0:1787826334.356080:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826334.356095:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977c6f100 x1874663346263040/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826334.356362:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346263040:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826334.356440:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.356478:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.356554:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346263040 00000100:00100000:1.0:1787826334.356563:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346263040 00000100:00100000:1.0:1787826334.356649:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23856 00000100:00100000:1.0:1787826334.356659:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346263040:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826334.357510:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346263040:12345-0@lo:1000:osp_up0-1.0 Request processed in 849us (1066us total) trans 8589957428 rc 0/0 00000100:00100000:1.0:1787826334.357520:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23856 00000100:00100000:3.0:1787826334.357980:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346263040:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787826334.358337:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e52:0x0]: returning size 4096 00010000:00010000:0.0:1787826334.358365:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358375:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358382:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358388:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826334.358393:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358399:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358414:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x197:0x0].0x0 (ffff9f7961f9a800) 00010000:00010000:0.0:1787826334.358417:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826334.358421:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358428:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b560 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358453:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358462:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358468:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358530:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826334.358541:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x195:0x0] 00010000:00010000:0.0:1787826334.358550:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358556:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358561:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358565:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826334.358568:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358574:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358583:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:0.0:1787826334.358585:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826334.358589:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358594:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4b559 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358604:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358611:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.358617:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.358647:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826334.358655:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.358667:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826334.358672:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826334.358684:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.358717:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.358723:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826334.358842:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346263168:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826334.358919:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.359018:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346263168 00000100:00100000:2.0:1787826334.359025:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346263168 00000100:00100000:2.0:1787826334.359075:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11141 00000100:00100000:2.0:1787826334.359085:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3653:x1874663346263168:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826334.359096:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826334.359131:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b56e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b567 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.359144:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x0 (ffff9f794385a100) 00010000:00010000:2.0:1787826334.359151:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826334.359192:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+14:3653:x1874663346263168:12345-0@lo:103:ldlm_bl.0 Request processed in 109us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.359198:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11141 00000100:00100000:2.0:1787826334.359248:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346263168:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826334.359910:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.359919:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68400/0xe21309aa81f4b567 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x195:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b56e expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.359938:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826334.359942:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.359948:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826334.359953:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826334.359961:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 5/0,0 mode: EX/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.359990:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.359996:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826334.360210:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:1.0:1787826334.360229:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.360238:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.360240:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.360242:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826334.360399:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1787826334.360405:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4b54b lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b552 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826334.360435:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826334.360481:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826334.360520:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346263296:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826334.360568:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826334.360639:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.360648:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.360655:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.360704:0:35092:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826334.360710:0:35092:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826334.360716:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.360724:0:35092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826334.360809:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225593:x1874663332630912:12345-192.168.203.56@tcp:36:rm.0 Request processed in 49331us (49538us total) trans 4294985434 rc 0/0 00000100:00100000:2.0:1787826334.360817:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68528 00000100:00100000:0.0:1787826334.365885:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346263296 00000100:00100000:0.0:1787826334.365902:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346263296 00000100:00100000:0.0:1787826334.365972:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11142 00000100:00100000:0.0:1787826334.365983:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+14:3655:x1874663346263296:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826334.366001:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826334.366196:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b552 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e52:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b54b expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.366219:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826334.366232:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826334.366511:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346263296:12345-0@lo:103:ldlm_bl.0 Request processed in 529us (5940us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.366519:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11142 00010000:00010000:1.0:1787826334.370151:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826334.370168:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.370182:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826334.370190:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826334.370201:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.370261:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.370269:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826334.370295:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.370307:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4b52f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b536 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.370325:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.370327:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.370329:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826334.370605:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826334.370632:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346263424:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826334.370715:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.370875:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346263424 00000100:00100000:1.0:1787826334.370883:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346263424 00000100:00100000:1.0:1787826334.370949:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11143 00000100:00100000:1.0:1787826334.370959:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346263424:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826334.370972:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826334.370994:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b536 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b52f expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.371342:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826334.371348:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826334.371449:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346263424:12345-0@lo:103:ldlm_bl.0 Request processed in 490us (732us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.371457:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11143 00000100:00100000:3.0:1787826334.376749:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346263296:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826334.379898:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346263424:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826334.439820:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.441129:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332631552 00000100:00100000:2.0:1787826334.441143:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332631552 00000100:00100000:2.0:1787826334.441211:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68529 00000100:00100000:2.0:1787826334.441221:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d24000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225601:x1874663332631552:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826334.441523:0:35080:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345561600 00000100:00100000:2.0:1787826334.441613:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d24000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225601:x1874663332631552:12345-192.168.203.56@tcp:41:stat.0 Request processed in 393us (1804us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.441621:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68529 00000100:00100000:3.0:1787826334.487284:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.487445:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332631680 00000100:00100000:2.0:1787826334.487456:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332631680 00000100:00100000:2.0:1787826334.487601:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68530 00000100:00100000:2.0:1787826334.487613:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225605:x1874663332631680:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826334.487636:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.487661:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b575 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080175f expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.487729:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.487745:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.487767:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.487819:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826334.487823:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826334.487845:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.487877:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b575 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.487896:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.487906:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826334.487909:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826334.487913:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826334.487985:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225605:x1874663332631680:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 374us (710us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.487992:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68530 00000100:00100000:1.0:1787826334.490608:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.490749:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332631808 00000100:00100000:2.0:1787826334.490774:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332631808 00000100:00100000:3.0:1787826334.491000:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68531 00000100:00100000:3.0:1787826334.491012:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225605:x1874663332631808:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826334.491031:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826334.491053:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b583 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801766 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.491148:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.491163:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.491175:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.491188:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.491200:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b544 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b53d expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.491209:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.491245:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b544 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b53d expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.491263:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6529:1874663346263552:0@lo:104: 00000100:00100000:3.0:1787826334.491323:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.491484:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826334.491529:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346263552 00000100:00100000:1.0:1787826334.491533:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346263552 00000100:00100000:1.0:1787826334.491570:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7888 00000100:00100000:1.0:1787826334.491578:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78437ac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6529:x1874663346263552:12345-0@lo:104: 00010000:00010000:1.0:1787826334.491594:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.491644:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826334.491650:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78437ac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6529:x1874663346263552:12345-0@lo:104: Request processed in 74us (328us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.491657:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7888 00010000:00010000:1.0:1787826334.491705:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826334.491710:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.491719:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826334.491726:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826334.491735:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.491773:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.491780:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826334.491803:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.491811:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b53d lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b544 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.491831:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.491833:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.491835:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826334.491848:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346263680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826334.491887:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.491939:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346263680 00000100:00100000:1.0:1787826334.491944:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346263680 00010000:00010000:1.0:1787826334.491962:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b544 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b53d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.491986:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21995 00000100:00100000:1.0:1787826334.491992:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346263680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826334.492001:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826334.493407:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.493452:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6529:1874663346263552:0@lo:104: 00010000:00010000:3.0:1787826334.493475:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826334.500949:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957428 is committed 00010000:00010000:1.0:1787826334.501160:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b544 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b53d expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.501185:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794385ad00) 00010000:00010000:1.0:1787826334.501192:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.501203:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.501211:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.501255:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826334.501356:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346263680:12345-0@lo:103:ldlm_bl.0 Request processed in 9362us (9467us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.501367:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21995 00010000:00010000:3.0:1787826334.501468:0:6529:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501477:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501559:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501568:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501575:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501582:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501589:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826334.501594:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501601:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501613:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826334.501616:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.501620:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501627:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b58a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501640:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501646:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501652:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501658:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826334.501660:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501666:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501674:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794385ad00) 00010000:00010000:3.0:1787826334.501677:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.501680:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501686:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b591 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501730:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b583 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801766 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501741:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826334.501743:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.501747:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b583 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801766 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.501757:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:2.0:1787826334.501898:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346263680:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826334.501954:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225605:x1874663332631808:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 10943us (11356us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826334.501963:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68531 00000100:00100000:1.0:1787826334.510932:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826334.511017:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332631936 00000100:00100000:3.0:1787826334.511028:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332631936 00000100:00100000:3.0:1787826334.511155:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68532 00000100:00100000:3.0:1787826334.511167:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225605:x1874663332631936:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826334.511188:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826334.511212:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b598 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080176d expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.511282:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.511299:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.511311:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826334.511364:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826334.511368:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826334.511403:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.511429:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b598 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.511447:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.511462:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826334.511465:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826334.511470:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826334.511554:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225605:x1874663332631936:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 389us (633us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.511562:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68532 00000100:00100000:1.0:1787826334.514580:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.514690:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332632192 00000100:00100000:2.0:1787826334.514701:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332632192 00000100:00100000:2.0:1787826334.514789:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68533 00000100:00100000:2.0:1787826334.514801:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225605:x1874663332632192:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:2.0:1787826334.514921:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346263808:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787826334.514981:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.515024:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826334.515238:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346263808 00000100:00100000:0.0:1787826334.515243:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346263808 00000100:00100000:0.0:1787826334.515285:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23857 00000100:00100000:0.0:1787826334.515293:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346263808:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787826334.515437:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346263808:12345-0@lo:1000:mdt00_003.0 Request processed in 143us (454us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.515445:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23857 00000100:00080000:1.0:1787826334.515513:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826334.515618:0:8752:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957428 gen 2 00000004:00080000:3.0:1787826334.515624:0:8752:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957426 committed transno 8589957428 00000004:00080000:3.0:1787826334.515643:0:8752:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957427 committed transno 8589957428 00000004:00080000:3.0:1787826334.515657:0:8752:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957428 committed transno 8589957428 00000100:00100000:3.0:1787826334.515672:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346263808:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.515721:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346263936:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.515799:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.515917:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826334.516071:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346263936 00000100:00100000:2.0:1787826334.516077:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346263936 00000100:00100000:2.0:1787826334.516157:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23858 00000100:00100000:2.0:1787826334.516167:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346263936:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787826334.516415:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346263936:12345-0@lo:1000:mdt00_003.0 Request processed in 249us (615us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826334.516423:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23858 00000100:00100000:3.0:1787826334.516703:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346263936:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787826334.517170:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.517182:0:8752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.517193:0:8752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.517212:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264064:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787826334.517402:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.517437:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826334.517553:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346264064 00000100:00100000:0.0:1787826334.517560:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346264064 00000100:00100000:0.0:1787826334.517612:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23859 00000100:00100000:0.0:1787826334.517620:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346264064:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787826334.517633:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826334.517648:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b5a6 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.517675:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.517682:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.517695:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.517719:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.517731:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 7 pid: 11629 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826334.517741:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346264192:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826334.518069:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.518080:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 6 pid: 6529 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826334.518091:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346264320:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826334.518234:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826334.520697:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346264192:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826334.527180:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826334.527394:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332632320 00000100:00100000:3.0:1787826334.527406:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332632320 00010000:00010000:3.0:1787826334.527462:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 8 pid: 11629 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.527515:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21996 00000100:00100000:3.0:1787826334.527525:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332632320:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826334.527539:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826334.527547:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 8 pid: 11629 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.527559:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 8 pid: 11629 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.527580:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b59f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080176d expref: 8 pid: 11629 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.527596:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826334.527602:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.527612:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826334.527678:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332632320:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 154us (510us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.527685:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21996 00000100:00100000:2.0:1787826334.527983:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.528079:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346264320:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826334.528128:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.528435:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826334.528440:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826334.528445:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.528454:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826334.528524:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:8752:x1874663346264064:12345-0@lo:101:mdt00_003.0 Request processed in 10905us (11122us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.528532:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23859 00000100:00100000:2.0:1787826334.528617:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264064:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787826334.528634:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.528643:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.529072:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332632448 00000100:00100000:3.0:1787826334.529079:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332632448 00010000:00010000:3.0:1787826334.529132:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 7 pid: 6529 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.529157:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 21997 00000100:00100000:3.0:1787826334.529166:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332632448:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826334.529177:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826334.529183:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 7 pid: 6529 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.529194:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 7 pid: 6529 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.529212:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b57c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080175f expref: 7 pid: 6529 timeout: 8246 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.529224:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826334.529228:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.529236:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.529243:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.530087:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.530174:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346264448:0@lo:105: 00000100:00100000:3.0:1787826334.530233:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.530352:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826334.531927:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346264448 00000100:00100000:2.0:1787826334.531941:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346264448 00000100:00100000:2.0:1787826334.532384:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7889 00000100:00100000:2.0:1787826334.532396:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346264448:12345-0@lo:105: 00010000:00010000:2.0:1787826334.532410:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.532420:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.532428:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.532437:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.532626:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.532634:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.532701:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68e00) 00000100:00100000:2.0:1787826334.532804:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346264448:12345-0@lo:105: Request processed in 412us (2570us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.532811:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7889 00000100:00100000:0.0:1787826334.532980:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346264448:0@lo:105: 00010000:00010000:0.0:1787826334.533001:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826334.533057:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332632448:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3890us (5073us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.533066:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 21997 00010000:00010000:3.0:1787826334.535075:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.535091:0:8752:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826334.535373:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826334.535423:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.535432:0:8752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.535519:0:8752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.535609:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264576:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787826334.535676:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.535784:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.535882:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346264576 00000100:00100000:1.0:1787826334.535890:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346264576 00000100:00100000:2.0:1787826334.545084:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23860 00000100:00100000:2.0:1787826334.545220:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:8752:x1874663346264576:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787826334.545245:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.546070:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b5bb lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b5b4 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.547203:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b5bb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b5b4 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.547227:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b5bb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b5b4 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.547236:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961f9a600) 00010000:00010000:2.0:1787826334.547239:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826334.547243:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826334.548810:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346264576:12345-0@lo:101:mdt00_003.0 Request processed in 3703us (13124us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.548822:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23860 00000100:00100000:3.0:1787826334.554832:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264576:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787826334.554866:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.554881:0:8752:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826334.554907:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826334.554967:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264704:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.555034:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.555082:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826334.555301:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346264704 00000100:00100000:2.0:1787826334.555309:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346264704 00000100:00100000:2.0:1787826334.555563:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23861 00000100:00100000:2.0:1787826334.555573:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346264704:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787826334.555658:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346264704:12345-0@lo:1000:mdt00_003.0 Request processed in 86us (623us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.555666:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23861 00000100:00100000:3.0:1787826334.555885:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854627800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264704:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.555935:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d817800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264832:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.555963:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.556027:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.559859:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346264832 00000100:00100000:1.0:1787826334.559876:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346264832 00000100:00100000:1.0:1787826334.559952:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23862 00000100:00100000:1.0:1787826334.559962:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346264832:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826334.560236:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346264832:12345-0@lo:1000:mdt00_003.0 Request processed in 275us (4271us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826334.560246:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23862 00000100:00100000:3.0:1787826334.563651:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d817800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264832:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.564027:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d817800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264960:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.564100:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.564205:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826334.564254:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346264960 00000100:00100000:2.0:1787826334.564261:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346264960 00000100:00100000:2.0:1787826334.564433:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23863 00000100:00100000:2.0:1787826334.564444:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346264960:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787826334.564586:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346264960:12345-0@lo:1000:mdt00_003.0 Request processed in 143us (484us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826334.564594:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23863 00000100:00100000:3.0:1787826334.564660:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d817800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346264960:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.564712:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346265088:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.564738:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.564966:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.565064:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346265088 00000100:00100000:1.0:1787826334.565070:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346265088 00000100:00100000:1.0:1787826334.565135:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23864 00000100:00100000:1.0:1787826334.565144:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346265088:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826334.565203:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346265088:12345-0@lo:1000:mdt00_003.0 Request processed in 61us (465us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.565210:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23864 00000100:00100000:3.0:1787826334.565422:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346265088:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.565480:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346265216:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787826334.565517:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.565538:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.565639:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346265216 00000100:00100000:1.0:1787826334.565644:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346265216 00000100:00100000:1.0:1787826334.565675:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23865 00000100:00100000:1.0:1787826334.565682:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346265216:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826334.565750:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346265216:12345-0@lo:1000:mdt00_003.0 Request processed in 68us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.565827:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23865 00000100:00100000:1.0:1787826334.565858:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346265216:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826334.566091:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346265344:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826334.566149:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.566164:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78474d26c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.566176:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346265344 00000100:00100000:1.0:1787826334.566180:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346265344 00000100:00100000:1.0:1787826334.566198:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23866 00000100:00100000:1.0:1787826334.566204:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346265344:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826334.566252:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:8752:x1874663346265344:12345-0@lo:1000:mdt00_003.0 Request processed in 48us (103us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.566258:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23866 00000100:00100000:1.0:1787826334.566533:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346265344:0@lo:1000:mdt00_003.0 00000040:00080000:1.0:1787826334.567865:0:8752:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1214, 2168 off1107120 00000040:00080000:1.0:1787826334.567952:0:8752:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1533, 2368 off1551040 00000004:00080000:1.0:1787826334.567957:0:8752:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784f58b840 with version 6855 00000004:00080000:3.0:1787826334.568194:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6855 rpc_version 6855 00000020:00100000:3.0:1787826334.568230:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826334.568239:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f797c320380 x1874663346265600/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826334.568259:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346265600:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826334.568334:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.568359:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.568471:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346265600 00000100:00100000:1.0:1787826334.568478:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346265600 00000100:00100000:1.0:1787826334.568534:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23867 00000100:00100000:1.0:1787826334.568542:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346265600:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826334.569721:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346265600:12345-0@lo:1000:osp_up0-1.0 Request processed in 1176us (1383us total) trans 8589957429 rc 0/0 00000100:00100000:1.0:1787826334.569731:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23867 00000100:00100000:2.0:1787826334.569987:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346265600:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826334.570930:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.570947:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.570964:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.570985:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.570991:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.570997:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.571090:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.571098:0:8752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.571354:0:8752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.571382:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98f480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346265728:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787826334.571440:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826334.571902:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d9b1540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826334.571959:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346265728 00000100:00100000:2.0:1787826334.571967:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346265728 00000100:00100000:2.0:1787826334.572071:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23868 00000100:00100000:2.0:1787826334.572081:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:8752:x1874663346265728:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787826334.572101:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.572140:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b5d7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b5d0 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.572162:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b5d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b5d0 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.572178:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b5d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b5d0 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.572185:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7961f9a700) 00010000:00010000:2.0:1787826334.572188:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826334.572191:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826334.572243:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:8752:x1874663346265728:12345-0@lo:101:mdt00_003.0 Request processed in 164us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.572249:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23868 00000100:00100000:1.0:1787826334.572324:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98f480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346265728:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787826334.572339:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572349:0:8752:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826334.572376:0:8752:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003b7908fb reply state 000000005643d830 transno 4294985435 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572387:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572392:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:1.0:1787826334.572394:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826334.572398:0:8752:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826334.572403:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572410:0:8752:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826334.572421:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x196:0x0] 00000004:00010000:1.0:1787826334.572427:0:8752:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003b7908fb reply state 000000005643d830 transno 4294985435 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572434:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572439:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:1.0:1787826334.572441:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826334.572442:0:8752:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787826334.572459:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e54:0x0]: returning size 4096 00000004:00080000:1.0:1787826334.572469:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826334.572479:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572499:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572505:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572533:0:8752:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826334.572538:0:8752:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826334.572542:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.572549:0:8752:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.572634:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225605:x1874663332632192:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 57834us (58063us total) trans 4294985435 rc 0/0 00000100:00100000:1.0:1787826334.572642:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68533 00010000:00010000:0.0:1787826334.572888:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826334.572896:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.572904:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826334.572910:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826334.572918:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.572947:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.572954:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826334.572984:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.572991:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b5a6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b5ad expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.573004:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826334.573006:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826334.573007:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826334.573016:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826334.573028:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346265856:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826334.573074:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826334.573143:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346265856 00000100:00100000:0.0:1787826334.573148:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346265856 00000100:00100000:0.0:1787826334.573186:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11144 00000100:00100000:0.0:1787826334.573194:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346265856:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826334.573204:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826334.573221:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b5ad lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b5a6 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.573233:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826334.573237:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826334.573265:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346265856:12345-0@lo:103:ldlm_bl.0 Request processed in 72us (191us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.573272:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11144 00000100:00100000:0.0:1787826334.573314:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346265856:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826334.590739:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.590959:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332632576 00000100:00100000:2.0:1787826334.590969:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332632576 00000100:00100000:2.0:1787826334.591036:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68534 00000100:00100000:2.0:1787826334.591047:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225594:x1874663332632576:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826334.591068:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.591091:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b5de lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801774 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.591213:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b5e5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.591228:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b5e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.591240:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b5e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.591288:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826334.591292:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826334.591313:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b5e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.591343:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b5de lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801774 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.591360:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b5e5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801774 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.591367:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826334.591370:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826334.591373:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826334.591437:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225594:x1874663332632576:12345-192.168.203.56@tcp:101:bash.0 Request processed in 391us (707us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.592001:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68534 00000100:00100000:3.0:1787826334.595016:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.595208:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332632704 00000100:00100000:2.0:1787826334.595218:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332632704 00000100:00100000:2.0:1787826334.595285:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68535 00000100:00100000:2.0:1787826334.595296:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225594:x1874663332632704:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826334.595315:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.595342:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b5ec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080177b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.595410:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.595424:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.595434:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.595457:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.595468:0:6527:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b5bb lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b5b4 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.595476:0:6527:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.595522:0:6527:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b5bb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b5b4 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826334.595540:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346265984:0@lo:104: 00000100:00100000:2.0:1787826334.595591:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.595640:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.595795:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346265984 00000100:00100000:1.0:1787826334.595800:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346265984 00000100:00100000:1.0:1787826334.595841:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7890 00000100:00100000:1.0:1787826334.595849:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346265984:12345-0@lo:104: 00010000:00010000:1.0:1787826334.595862:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.595925:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826334.595931:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346265984:12345-0@lo:104: Request processed in 84us (340us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.595938:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7890 00010000:00010000:0.0:1787826334.595982:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826334.595990:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.595999:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826334.596005:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826334.596013:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.596042:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.596049:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826334.596083:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.596091:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69400/0xe21309aa81f4b5b4 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b5bb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.596138:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826334.596140:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826334.596142:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826334.596193:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826334.596206:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346266112:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826334.596242:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826334.596368:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346266112 00000100:00100000:0.0:1787826334.596373:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346266112 00010000:00010000:0.0:1787826334.596392:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b5bb lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b5b4 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826334.596422:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21998 00000100:00100000:0.0:1787826334.596429:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346266112:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826334.596441:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787826334.597411:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346265984:0@lo:104: 00010000:00010000:2.0:1787826334.597434:0:6527:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826334.600568:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957429 is committed 00010000:00010000:1.0:1787826334.604076:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4b5bb lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b5b4 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.604101:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961f9a600) 00010000:00010000:1.0:1787826334.604125:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.604136:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.604144:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.604360:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826334.604455:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346266112:12345-0@lo:103:ldlm_bl.0 Request processed in 8024us (8210us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.604465:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21998 00000100:00100000:0.0:1787826334.605994:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346266112:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826334.606087:0:6527:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.606099:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826334.606278:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0f480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6527:1874663346266240:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826334.606351:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826334.606388:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7844e81a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.606397:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346266240 00000100:00100000:1.0:1787826334.606406:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346266240 00000100:00100000:1.0:1787826334.606460:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23869 00000100:00100000:1.0:1787826334.606472:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6527:x1874663346266240:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826334.606852:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6527:x1874663346266240:12345-0@lo:1000:mdt00_000.0 Request processed in 381us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.606860:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23869 00000100:00100000:3.0:1787826334.607303:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0f480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6527:1874663346266240:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1787826334.607374:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b601 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607389:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b601 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607399:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b601 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826334.607422:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b601 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607436:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607443:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607452:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607459:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607465:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826334.607469:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607475:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607496:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826334.607500:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.607505:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607511:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b5f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607524:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607529:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607534:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607539:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826334.607542:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607548:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607556:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7961f9a600) 00010000:00010000:3.0:1787826334.607558:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826334.607562:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607566:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b5fa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607608:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4b5ec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080177b expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607623:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b601 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080177b expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826334.607630:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826334.607633:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826334.607635:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826334.607695:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225594:x1874663332632704:12345-192.168.203.56@tcp:101:bash.0 Request processed in 12402us (12689us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.607702:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68535 00000100:00100000:1.0:1787826334.612033:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.612155:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332632832 00000100:00100000:0.0:1787826334.612164:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332632832 00000100:00100000:0.0:1787826334.612221:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68536 00000100:00100000:0.0:1787826334.612231:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225594:x1874663332632832:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826334.612257:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826334.612285:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b608 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801782 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.612373:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b60f lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.612387:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b60f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.612397:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b60f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826334.612522:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e54:0x0]: returning size 4096 00000004:00200000:0.0:1787826334.612527:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826334.612569:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b608 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801782 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.612587:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b60f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801782 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.612601:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:0.0:1787826334.612604:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826334.612608:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826334.612676:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225594:x1874663332632832:12345-192.168.203.56@tcp:101:bash.0 Request processed in 446us (653us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.612682:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68536 00000100:00100000:1.0:1787826334.618550:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.618658:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332632960 00000100:00100000:0.0:1787826334.618668:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332632960 00000100:00100000:0.0:1787826334.618737:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68537 00000100:00100000:0.0:1787826334.618748:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225594:x1874663332632960:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826334.618772:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826334.618796:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b616 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801789 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.618861:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4b61d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.618875:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4b61d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.618887:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4b61d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826334.618937:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x198:0x0]: returning size 4096 00000004:00200000:0.0:1787826334.618942:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826334.618980:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b616 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801789 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.619005:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4b61d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801789 expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.619260:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:0.0:1787826334.619264:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826334.619268:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:1.0:1787826334.619563:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225594:x1874663332632960:12345-192.168.203.56@tcp:101:bash.0 Request processed in 815us (1022us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.619570:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68537 00000100:00100000:3.0:1787826334.631410:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.631721:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332633088 00000100:00100000:2.0:1787826334.631737:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332633088 00000100:00100000:2.0:1787826334.631835:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68538 00000100:00100000:2.0:1787826334.631847:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225594:x1874663332633088:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826334.631869:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.631904:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b624 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801790 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.631971:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.631985:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b5d7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b5d0 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.631994:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 4/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.632028:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b5d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b5d0 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826334.632043:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6528:1874663346266368:0@lo:104: 00000100:00100000:2.0:1787826334.632150:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.632201:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826334.632514:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346266368 00000100:00100000:1.0:1787826334.632519:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346266368 00000100:00100000:1.0:1787826334.632567:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7891 00000100:00100000:1.0:1787826334.632575:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6528:x1874663346266368:12345-0@lo:104: 00010000:00010000:1.0:1787826334.632589:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.632646:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826334.632652:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6528:x1874663346266368:12345-0@lo:104: Request processed in 79us (502us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.632659:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7891 00010000:00010000:1.0:1787826334.632678:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826334.632684:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.632692:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826334.632698:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826334.632707:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.632731:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.632737:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826334.633223:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6528:1874663346266368:0@lo:104: 00010000:00010000:0.0:1787826334.633245:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826334.633447:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346266496:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826334.633506:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826334.633600:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.633610:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece000/0xe21309aa81f4b5d0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b5d7 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.633635:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.633637:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826334.633639:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826334.634440:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346266496 00000100:00100000:0.0:1787826334.634449:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346266496 00010000:00010000:1.0:1787826334.634566:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b5d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b5d0 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826334.634635:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21999 00000100:00100000:1.0:1787826334.634645:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346266496:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826334.634661:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826334.635194:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4b5d7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b5d0 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.635211:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7961f9a700) 00010000:00010000:1.0:1787826334.635217:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.635226:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.635233:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.635310:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826334.635383:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346266496:12345-0@lo:103:ldlm_bl.0 Request processed in 740us (1876us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.635392:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21999 00000100:00100000:3.0:1787826334.635472:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346266496:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826334.635605:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.635615:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826334.635677:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x196:0x0]: returning size 4096 00000004:00200000:2.0:1787826334.635682:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826334.635726:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b624 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801790 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.635762:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801790 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.635771:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7961f9a700) 00010000:00010000:2.0:1787826334.635774:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826334.635777:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826334.635836:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225594:x1874663332633088:12345-192.168.203.56@tcp:101:bash.0 Request processed in 3991us (4512us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.635844:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68538 00000100:00100000:3.0:1787826334.649660:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.649782:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332633344 00000100:00100000:2.0:1787826334.649799:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332633344 00000100:00100000:2.0:1787826334.649885:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68539 00000100:00100000:2.0:1787826334.649896:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225594:x1874663332633344:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826334.649917:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.649920:0:11629:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826334.649948:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b62b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801790 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.649969:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7961f9a700) 00010000:00010000:2.0:1787826334.649982:0:11629:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826334.649998:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b632 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801797 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.650189:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.650206:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.650214:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.650225:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.650232:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.650237:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826334.651484:0:11629:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x521:0x0], index 0: 0 00010000:00010000:2.0:1787826334.652460:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b647 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652480:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b647 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652501:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b647 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652541:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652548:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652553:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652558:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652564:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826334.652568:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652575:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652587:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7961f9aa00) 00010000:00010000:2.0:1787826334.652589:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826334.652593:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.652599:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b639 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826334.655019:0:11629:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008145de9a reply state 00000000ffce5da9 transno 8589957430 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.655050:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.655058:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f7961f9a600) 00010000:00010000:2.0:1787826334.655061:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826334.655065:0:11629:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826334.655914:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4b632 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801797 expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.655953:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b647 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801797 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.655961:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f7961f9a800) 00010000:00010000:2.0:1787826334.655964:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826334.655966:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826334.656048:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225594:x1874663332633344:12345-192.168.203.56@tcp:101:bash.0 Request processed in 6153us (6400us total) trans 8589957430 rc 0/0 00000100:00100000:2.0:1787826334.656057:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68539 00000100:00100000:3.0:1787826334.661898:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.662029:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332633472 00000100:00100000:0.0:1787826334.662047:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332633472 00000100:00100000:0.0:1787826334.662180:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30852 00000100:00100000:0.0:1787826334.662191:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225594:x1874663332633472:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826334.662210:0:20167:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826334.662233:0:20167:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080179e expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826334.662268:0:20167:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x521:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00000100:00080000:1.0:1787826334.662376:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787826334.662598:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=8146s v=(0 0) 00010000:00010000:0.0:1787826334.662604:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826334.662615:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 7 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.662640:0:20167:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 7 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826334.662653:0:20167:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826334.662756:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225594:x1874663332633472:12345-192.168.203.56@tcp:101:bash.0 Request processed in 567us (871us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.662764:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30852 00000100:00100000:3.0:1787826334.676457:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.676628:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332633728 00000100:00100000:0.0:1787826334.676638:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332633728 00000100:00100000:0.0:1787826334.676706:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9366 00000100:00100000:0.0:1787826334.676717:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225594:x1874663332633728:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826334.677030:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225594:x1874663332633728:12345-192.168.203.56@tcp:35:bash.0 Request processed in 313us (591us total) trans 8589957431 rc 0/0 00000100:00100000:0.0:1787826334.677037:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9366 00000100:00100000:3.0:1787826334.685145:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.685781:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332633984 00000100:00100000:2.0:1787826334.685793:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332633984 00000100:00100000:2.0:1787826334.685853:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68540 00000100:00100000:2.0:1787826334.685865:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225594:x1874663332633984:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826334.685891:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.685894:0:35095:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826334.685924:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4b61d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801789 expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.685944:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:2.0:1787826334.685949:0:35095:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826334.685964:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b65c lrc: 2/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017a5 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.686092:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.686152:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.686163:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.686178:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.686185:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.686191:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826334.686731:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346266624:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826334.686857:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826334.687023:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826334.687129:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346266624 00000100:00100000:3.0:1787826334.687136:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346266624 00000100:00100000:3.0:1787826334.687178:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23870 00000100:00100000:3.0:1787826334.687188:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346266624:12345-0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826334.687274:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346266624:12345-0@lo:1000:mdt00_012.0 Request processed in 88us (417us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826334.687291:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23870 00000100:00100000:2.0:1787826334.687510:0:35095:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957429 gen 2 00000004:00080000:2.0:1787826334.687516:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957429 committed transno 8589957429 00000100:00100000:2.0:1787826334.687544:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346266624:0@lo:1000:mdt00_012.0 00000004:00080000:2.0:1787826334.688542:0:35095:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4cc:0x0], index 1: 0 00010000:00010000:2.0:1787826334.689089:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b671 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689125:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b671 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689138:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b671 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689176:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689184:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689191:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689198:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689203:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826334.689208:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689215:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689226:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:2.0:1787826334.689229:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826334.689233:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689239:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b663 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826334.689262:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e33e80f6 reply state 00000000fcdc1d54 transno 4294985436 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689274:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689453:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x4ec94fd6 (ffff9f7961f9aa00) 00010000:00010000:2.0:1787826334.689456:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826334.689460:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826334.689508:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4b65c lrc: 0/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017a5 expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689523:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b671 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017a5 expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.689531:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f7961f9a700) 00010000:00010000:2.0:1787826334.689534:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826334.689537:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826334.690673:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225594:x1874663332633984:12345-192.168.203.56@tcp:101:bash.0 Request processed in 4802us (5565us total) trans 4294985436 rc 0/0 00000100:00100000:2.0:1787826334.690684:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68540 00000100:00100000:3.0:1787826334.707809:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.707905:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332634112 00000100:00100000:2.0:1787826334.707915:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332634112 00000100:00100000:2.0:1787826334.707994:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30853 00000100:00100000:2.0:1787826334.708006:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:225594:x1874663332634112:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826334.708027:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826334.708051:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017ac expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826334.708080:0:17826:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4cc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826334.708102:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cc:0x0].0x0 since=8146s v=(0 0) 00010000:00010000:2.0:1787826334.708139:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826334.708150:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.708169:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.708183:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826334.708256:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225594:x1874663332634112:12345-192.168.203.56@tcp:101:bash.0 Request processed in 242us (448us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.708264:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30853 00000100:00080000:1.0:1787826334.708769:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826334.713992:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.714523:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332634368 00000100:00100000:0.0:1787826334.714534:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332634368 00000100:00100000:0.0:1787826334.714601:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9367 00000100:00100000:0.0:1787826334.714613:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225594:x1874663332634368:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826334.714952:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225594:x1874663332634368:12345-192.168.203.56@tcp:35:bash.0 Request processed in 340us (980us total) trans 4294985437 rc 0/0 00000100:00100000:0.0:1787826334.714960:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9367 00000100:00100000:2.0:1787826334.731394:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.731476:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826334.731645:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826334.733023:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332634496 00000100:00100000:1.0:1787826334.733037:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332634496 00000100:00100000:0.0:1787826334.735695:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332634624 00000100:00100000:0.0:1787826334.735710:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332634624 00000100:00100000:1.0:1787826334.735949:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332634752 00000100:00100000:1.0:1787826334.735954:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332634752 00000100:00100000:2.0:1787826334.737706:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11145 00000100:00100000:2.0:1787826334.737729:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79468f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:34880:x1874663332634496:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826334.737749:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826334.737787:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b5e5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801774 expref: 12 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.737810:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826334.737822:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b647 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801797 expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.737832:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f7961f9a800) 00010000:00010000:2.0:1787826334.737851:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b601 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080177b expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.737865:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:2.0:1787826334.737871:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826334.737940:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79468f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:34880:x1874663332634496:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 221us (6559us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.737948:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11145 00000100:00100000:1.0:1787826334.737953:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11146 00000100:00100000:1.0:1787826334.737963:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:49149:x1874663332634752:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826334.737971:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826334.737983:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b46b lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dbd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801727 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826334.737992:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbd:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:1.0:1787826334.737999:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826334.738008:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11147 00000100:00100000:2.0:1787826334.738015:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:6166:x1874663332634624:12345-192.168.203.56@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826334.738017:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:49149:x1874663332634752:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 58us (6374us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.738022:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11146 00010000:00010000:2.0:1787826334.738024:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826334.738036:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b671 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017a5 expref: 13 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.738047:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f7961f9a700) 00010000:00010000:2.0:1787826334.738059:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b60f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801782 expref: 12 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.738068:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:2.0:1787826334.738076:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b3ca lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e52:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801704 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826334.738084:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e52:0x0].0x0 (ffff9f784b4a5e00) 00010000:00010000:2.0:1787826334.738091:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826334.738134:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:6166:x1874663332634624:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 119us (6658us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.738140:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11147 00010000:00010000:0.0:1787826334.893579:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826334.939513:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826334.939622:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826334.939665:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826334.939722:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332634880 00000100:00100000:0.0:1787826334.939731:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332634880 00000100:00100000:0.0:1787826334.939799:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2230 00000100:00100000:0.0:1787826334.939810:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd93c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332634880:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826334.939884:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd93c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332634880:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 76us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826334.939892:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2230 00000100:00100000:2.0:1787826334.943019:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332635008 00000100:00100000:2.0:1787826334.943034:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332635008 00000100:00100000:1.0:1787826334.943075:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332635136 00000100:00100000:1.0:1787826334.943080:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332635136 00000100:00100000:2.0:1787826334.943180:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8841 00000100:00100000:2.0:1787826334.943190:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332635008:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826334.943281:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332635008:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 91us (3780us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826334.943288:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8841 00000100:00080000:3.0:1787826334.943396:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826334.943400:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826334.943711:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8842 00000100:00100000:1.0:1787826334.943721:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332635136:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826334.945007:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332635136:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 1277us (5329us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826334.945019:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8842 00000100:00100000:3.0:1787826335.725692:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346266880:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.725776:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.725811:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346267392:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.725836:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.725855:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346267904:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.725877:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826335.725908:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346266880 00000100:00100000:3.0:1787826335.725908:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:3.0:1787826335.725916:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346266752:0@lo:400:kworker.0 00000100:00100000:1.0:1787826335.725918:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346266880 00000100:00100000:3.0:1787826335.725957:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826335.725965:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346267904 00000100:00100000:1.0:1787826335.725971:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346267904 00000100:00100000:3.0:1787826335.725997:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346267264:0@lo:400:kworker.0 00000100:00100000:1.0:1787826335.726002:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23871 00000100:00100000:1.0:1787826335.726015:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346266880:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726033:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.726050:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346267776:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726080:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826335.726085:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346266880:12345-0@lo:400:kworker.0 Request processed in 70us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826335.726094:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23871 00000100:00100000:3.0:1787826335.726098:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346268288:0@lo:400:kworker.0 00000100:00100000:1.0:1787826335.726143:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23872 00000100:00100000:3.0:1787826335.726148:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826335.726154:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346267904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726182:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346266880:0@lo:400:kworker.0 00000100:00100000:1.0:1787826335.726198:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346267904:12345-0@lo:400:kworker.0 Request processed in 46us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.726200:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346267904:0@lo:400:kworker.0 00000100:00100000:1.0:1787826335.726212:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23872 00000100:00100000:3.0:1787826335.726239:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346267008:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726280:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.726302:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346267520:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726326:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.726350:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346268032:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726372:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.726389:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1787826335.726394:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346267136:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726419:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.726433:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f2a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346267648:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726454:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.726471:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346268160:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726508:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826335.726546:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346267392 00000100:00100000:3.0:1787826335.726550:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346267392 00000100:00100000:3.0:1787826335.726588:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346267776 00000100:00100000:3.0:1787826335.726591:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346267776 00000100:00100000:3.0:1787826335.726612:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346268032 00000100:00100000:3.0:1787826335.726615:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346268032 00000100:00100000:3.0:1787826335.726633:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346268160 00000100:00100000:3.0:1787826335.726636:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346268160 00000100:00100000:3.0:1787826335.726658:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8843 00000100:00100000:3.0:1787826335.726666:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346267392:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726706:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346267392:12345-0@lo:400:kworker.0 Request processed in 42us (868us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.726714:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8843 00000100:00100000:3.0:1787826335.726728:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8844 00000100:00100000:3.0:1787826335.726735:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346267776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726764:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346267776:12345-0@lo:400:kworker.0 Request processed in 30us (685us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.726770:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8844 00000100:00100000:3.0:1787826335.726782:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8845 00000100:00100000:3.0:1787826335.726789:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346268032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.726822:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346268032:12345-0@lo:400:kworker.0 Request processed in 34us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.726827:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8845 00000100:00100000:3.0:1787826335.726838:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8846 00000100:00100000:3.0:1787826335.726845:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346268160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826335.726870:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346267264 00000100:00100000:3.0:1787826335.726877:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346268160:12345-0@lo:400:kworker.0 Request processed in 32us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826335.726878:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346267264 00000100:00100000:3.0:1787826335.726886:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8846 00000100:00100000:3.0:1787826335.726914:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346266752 00000100:00100000:3.0:1787826335.726918:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346266752 00000100:00100000:0.0:1787826335.726925:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346268288 00000100:00100000:0.0:1787826335.726932:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346268288 00000100:00100000:3.0:1787826335.726953:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2231 00000100:00100000:3.0:1787826335.726963:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cd1dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346266752:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826335.726964:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346267008 00000100:00100000:0.0:1787826335.726970:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346267008 00000100:00100000:0.0:1787826335.727000:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346267520 00000100:00100000:3.0:1787826335.727003:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cd1dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346266752:12345-0@lo:400:kworker.0 Request processed in 41us (1044us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826335.727005:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346267520 00000100:00100000:3.0:1787826335.727014:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2231 00000100:00100000:0.0:1787826335.727049:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346267136 00000100:00100000:3.0:1787826335.727054:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346267648 00000100:00100000:0.0:1787826335.727055:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346267136 00000100:00100000:3.0:1787826335.727060:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346267648 00000100:00100000:0.0:1787826335.727090:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14328 00000100:00100000:0.0:1787826335.727101:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346267264:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727102:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14329 00000100:00100000:3.0:1787826335.727138:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346268288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727172:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851243100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346268288:12345-0@lo:400:kworker.0 Request processed in 37us (1025us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.727179:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14329 00000100:00100000:3.0:1787826335.727197:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14330 00000100:00100000:3.0:1787826335.727204:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346267008:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727234:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346267008:12345-0@lo:400:kworker.0 Request processed in 30us (954us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.727239:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14330 00000100:00100000:3.0:1787826335.727257:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14331 00000100:00100000:3.0:1787826335.727266:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346267520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826335.727315:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346267264:12345-0@lo:400:kworker.0 Request processed in 215us (1280us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.727316:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346267520:12345-0@lo:400:kworker.0 Request processed in 51us (989us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826335.727330:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14328 00000100:00100000:3.0:1787826335.727559:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14331 00000100:00100000:3.0:1787826335.727578:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14332 00000100:00100000:3.0:1787826335.727585:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346267136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727618:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346267136:12345-0@lo:400:kworker.0 Request processed in 35us (1199us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.727627:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14332 00000100:00100000:3.0:1787826335.727638:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14333 00000100:00100000:3.0:1787826335.727644:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346267648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727667:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346267648:12345-0@lo:400:kworker.0 Request processed in 24us (1213us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.727673:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14333 00000100:00080000:3.0:1787826335.727711:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787826335.727715:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826335.727718:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826335.727720:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826335.727760:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346267136:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727772:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f2a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346267648:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727782:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346268160:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727794:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346267008:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727803:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346267520:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727814:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973852 gen 2 00000004:00080000:3.0:1787826335.727819:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f785189ed80, transno 4294973852 00000100:00100000:3.0:1787826335.727841:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346268032:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727875:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b662680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346266752:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727882:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346267264:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727889:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346267776:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727895:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346268288:0@lo:400:kworker.0 00000100:00100000:3.0:1787826335.727902:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346267392:0@lo:400:kworker.0 00010000:00010000:0.0:1787826335.916328:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826335.954093:0:166637:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:0.0:1787826335.954164:0:166637:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000100:00100000:1.0:1787826335.974961:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826335.976096:0:24160:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332635264 00000100:00100000:2.0:1787826335.976131:0:24160:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332635264 00002000:00100000:2.0:1787826335.976193:0:24160:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_011: refresh rw locks for [0x280000401:0x521:0x0] (0->5) req@ffff9f784d98df80 x1874663332635264/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:691/0 lens 496/0 e 0 to 0 dl 1787826346 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826335.976206:0:24160:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d98df80 with x1874663332635264 ext(0->5) 00002000:00010000:2.0:1787826335.976212:0:24160:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784d98df80 00000100:00100000:2.0:1787826335.976244:0:24160:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27656 00000100:00100000:2.0:1787826335.976254:0:24160:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332635264:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826335.981456:0:24160:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332635264:12345-192.168.203.56@tcp:4:bash.0 Request processed in 5193us (6494us total) trans 4294973854 rc 0/0 00000100:00100000:2.0:1787826335.981468:0:24160:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27656 00002000:00100000:2.0:1787826335.981493:0:24160:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_011: refresh rw locks for [0x280000401:0x521:0x0] (0->5) req@ffff9f784d98df80 x1874663332635264/t4294973854(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:691/0 lens 496/448 e 0 to 0 dl 1787826346 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826335.981507:0:24160:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d98df80 with x1874663332635264 ext(0->5) 00002000:00010000:2.0:1787826335.981516:0:24160:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784d98df80 00000100:00100000:3.0:1787826335.990068:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826335.993908:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332635392 00000100:00100000:2.0:1787826335.993923:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332635392 00000100:00100000:2.0:1787826335.993993:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30854 00000100:00100000:2.0:1787826335.994005:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332635392:12345-192.168.203.56@tcp:16:kworker.0 00000001:00080000:0.0:1787826335.995988:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973854 is committed 00000100:00100000:3.0:1787826335.996424:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332635392:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 2415us (6940us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826335.996436:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30854 00000100:00100000:0.0:1787826336.013882:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826336.013981:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826336.014030:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332635648 00000100:00100000:3.0:1787826336.014041:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332635648 00000100:00100000:1.0:1787826336.014156:0:20174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332635520 00000100:00100000:1.0:1787826336.014161:0:20174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332635520 00000100:00100000:3.0:1787826336.014203:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30855 00002000:00100000:1.0:1787826336.014205:0:20174:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0x4cc:0x0] (0->5) req@ffff9f7977c6f100 x1874663332635520/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:692/0 lens 496/0 e 0 to 0 dl 1787826347 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826336.014216:0:20174:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977c6f100 with x1874663332635520 ext(0->5) 00000100:00100000:3.0:1787826336.014219:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332635648:12345-192.168.203.56@tcp:16:kworker.0 00002000:00010000:1.0:1787826336.014224:0:20174:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7977c6f100 00000100:00100000:1.0:1787826336.014250:0:20174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27657 00000100:00100000:1.0:1787826336.014260:0:20174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332635520:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:3.0:1787826336.014323:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332635648:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 109us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826336.014331:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30855 00000100:00080000:0.0:1787826336.014966:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826336.019362:0:20174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332635520:12345-192.168.203.56@tcp:4:bash.0 Request processed in 5093us (5479us total) trans 4294976164 rc 0/0 00000100:00100000:1.0:1787826336.019451:0:20174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27657 00002000:00100000:1.0:1787826336.019468:0:20174:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0x4cc:0x0] (0->5) req@ffff9f7977c6f100 x1874663332635520/t4294976164(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:692/0 lens 496/448 e 0 to 0 dl 1787826347 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826336.019489:0:20174:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7977c6f100 with x1874663332635520 ext(0->5) 00002000:00010000:1.0:1787826336.019496:0:20174:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7977c6f100 00000100:00100000:2.0:1787826336.236511:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826336.236538:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346268416:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826336.236628:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826336.236693:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346268416 00000100:00100000:2.0:1787826336.236701:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346268416 00000100:00100000:2.0:1787826336.236754:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6243 00000100:00100000:2.0:1787826336.236765:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346268416:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1787826336.236788:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826336.236836:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346268416:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826336.236853:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6243 00000100:00100000:0.0:1787826336.237098:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346268416:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826336.750432:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346268800:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826336.750494:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346268928:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826336.750543:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826336.750544:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1787826336.750596:0:8751:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8148962488918ns, next at 8149962485765ns, rc = 0 00000100:00100000:3.0:1787826336.750794:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346268928 00000100:00100000:3.0:1787826336.750801:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346268928 00000100:00100000:0.0:1787826336.750842:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346268544:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826336.750878:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826336.750900:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346269056:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826336.750933:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826336.750955:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826336.750962:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346268672:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826336.750981:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346268800 00000100:00100000:1.0:1787826336.750987:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346268800 00000100:00100000:0.0:1787826336.750990:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826336.751037:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346268672 00000100:00100000:1.0:1787826336.751041:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346268672 00000100:00100000:0.0:1787826336.751056:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346268544 00000100:00100000:0.0:1787826336.751062:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346268544 00000100:00100000:0.0:1787826336.751086:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346269056 00000100:00100000:0.0:1787826336.751089:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346269056 00000100:00100000:0.0:1787826336.751136:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6244 00000100:00100000:0.0:1787826336.751147:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346268544:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826336.751157:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23873 00000100:00100000:1.0:1787826336.751168:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346268800:12345-0@lo:41:osp-pre-1-0.0 00002000:00000024:0.0:1787826336.751171:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662426112 00000004:00000024:1.0:1787826336.751196:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443385344 avail 1297145856 00000100:00100000:0.0:1787826336.751219:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346268544:12345-0@lo:13:osp-pre-0-0.0 Request processed in 74us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826336.751227:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6244 00000100:00100000:1.0:1787826336.751248:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346268800:12345-0@lo:41:osp-pre-1-0.0 Request processed in 82us (708us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826336.751251:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6245 00000100:00100000:1.0:1787826336.751262:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23873 00000100:00100000:0.0:1787826336.751264:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346269056:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787826336.751276:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826336.751283:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23874 00000100:00100000:1.0:1787826336.751292:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346268672:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826336.751310:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346269056:12345-0@lo:13:osp-pre-1-0.0 Request processed in 49us (375us total) trans 0 rc 0/0 00000004:00000024:1.0:1787826336.751310:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442783232 avail 1296543744 00000100:00100000:0.0:1787826336.751322:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6245 00000100:00100000:1.0:1787826336.751353:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346268672:12345-0@lo:41:osp-pre-0-1.0 Request processed in 61us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826336.751372:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346268672:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826336.751385:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346268544:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826336.751396:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346269056:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826336.751753:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6246 00000100:00100000:3.0:1787826336.751762:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346268928:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826336.751773:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893977088 avail 3662426112 00000100:00100000:3.0:1787826336.751828:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346268928:12345-0@lo:13:osp-pre-0-1.0 Request processed in 54us (1274us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826336.751835:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6246 00000100:00100000:3.0:1787826336.751883:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346268928:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826336.751988:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23874 00010000:00010000:3.0:1787826336.754179:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b50c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826336.754201:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b50c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754243:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b50c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754250:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826336.754280:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4b505 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b50c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826336.754302:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985437 is committed 00000020:00080000:3.0:1787826336.754323:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784f58b180 refcount 2 committed 0 result 0 batchid 3429 00000020:00080000:3.0:1787826336.754327:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abffc00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f58b840 00000020:00080000:3.0:1787826336.754330:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1533 00000100:00100000:1.0:1787826336.754334:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346269184:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:3.0:1787826336.754335:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff200 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78525e4c00 00000020:00080000:3.0:1787826336.754343:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1214 00000001:00080000:3.0:1787826336.754356:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985434 is committed 00000020:00080000:3.0:1787826336.754360:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7844e81240 refcount 2 committed 0 result 0 batchid 3428 00000020:00080000:3.0:1787826336.754362:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e81a80 00000020:00080000:3.0:1787826336.754365:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1532 00000100:00100000:1.0:1787826336.754417:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826336.754484:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346269184 00000100:00100000:1.0:1787826336.754491:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346269184 00000100:00100000:1.0:1787826336.754539:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11148 00000100:00100000:1.0:1787826336.754550:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346269184:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826336.754563:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826336.754583:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b50c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x195:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b505 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.754599:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x195:0x0].0x5a734ce9 (ffff9f79785ccd00) 00010000:00010000:1.0:1787826336.754610:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000020:00080000:3.0:1787826336.754699:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78525e5600 00000020:00080000:3.0:1787826336.754702:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1213 00000004:00080000:3.0:1787826336.754713:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 366 committed 00000020:00080000:3.0:1787826336.754725:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7844e81540 refcount 2 committed 0 result 0 batchid 3427 00000020:00080000:3.0:1787826336.754728:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7844e81000 00000020:00080000:3.0:1787826336.754731:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1531 00000020:00080000:3.0:1787826336.754732:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7952192600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78492f6a00 00000020:00080000:3.0:1787826336.754735:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1212 00000020:00080000:3.0:1787826336.754741:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7849247180 refcount 2 committed 0 result 0 batchid 3426 00000020:00080000:3.0:1787826336.754743:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784a925200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7849247780 00000020:00080000:3.0:1787826336.754746:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1530 00000020:00080000:3.0:1787826336.754748:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784a925f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7847b18400 00000020:00080000:3.0:1787826336.754750:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1211 00000001:00080000:3.0:1787826336.754756:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985428 is committed 00000020:00080000:3.0:1787826336.754768:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3425 batchid updated 00010000:00010000:3.0:1787826336.754786:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754795:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754802:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754807:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826336.754874:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754884:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754899:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:3.0:1787826336.754902:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826336.754907:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754913:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b5c2 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754929:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826336.754930:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346269184:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 273us (404us total) trans 0 rc 0/0 00010000:00010000:3.0:1787826336.754944:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826336.754949:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11148 00010000:00010000:3.0:1787826336.754957:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754964:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826336.754967:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754974:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754982:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:3.0:1787826336.754985:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826336.754988:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.754993:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b5c9 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826336.755000:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346269184:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787826336.755010:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.755019:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.755025:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.755030:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826336.755034:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.755040:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.755049:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x4ec94fd6 (ffff9f7961f9aa00) 00010000:00010000:3.0:1787826336.755052:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826336.755055:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.755061:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b66a lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787826336.755086:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3429 committed batchid 3425 00000004:00080000:3.0:1787826336.755210:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784549fcc0 with version 6856 00000020:00080000:3.0:1787826336.755223:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3422 cancel update log [0x200000401:0x3:0x0].1526: rc = 0 00000004:00080000:1.0:1787826336.755256:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6856 rpc_version 6856 00000020:00080000:3.0:1787826336.755268:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3422 cancel update log [0x240000400:0x3:0x0].1207: rc = 0 00000020:00100000:1.0:1787826336.755281:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826336.755291:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784c6bb480 x1874663346269440/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000004:00080000:3.0:1787826336.755294:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784549f300 with version 6857 00000020:00080000:3.0:1787826336.755300:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3423 cancel update log [0x200000401:0x3:0x0].1527: rc = 0 00000100:00100000:1.0:1787826336.755319:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346269440:0@lo:1000:osp_up0-1.0 00000020:00080000:3.0:1787826336.755326:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3423 cancel update log [0x240000400:0x3:0x0].1208: rc = 0 00000004:00080000:3.0:1787826336.755347:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784549fd80 with version 6858 00000020:00080000:3.0:1787826336.755350:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3424 cancel update log [0x200000401:0x3:0x0].1528: rc = 0 00000020:00080000:3.0:1787826336.755425:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3424 cancel update log [0x240000400:0x3:0x0].1209: rc = 0 00000100:00100000:1.0:1787826336.755462:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:3.0:1787826336.755663:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78450263c0 with version 6859 00000020:00080000:3.0:1787826336.755671:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3425 cancel update log [0x200000401:0x3:0x0].1529: rc = 0 00000100:00100000:1.0:1787826336.755674:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346269440 00000100:00100000:1.0:1787826336.755681:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346269440 00000020:00080000:3.0:1787826336.755710:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3425 cancel update log [0x240000400:0x3:0x0].1210: rc = 0 00000100:00100000:1.0:1787826336.755717:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23875 00000020:00080000:3.0:1787826336.755719:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3425 00000100:00100000:1.0:1787826336.755728:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346269440:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826336.755785:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826336.755795:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346269568:0@lo:6:osp-syn-0-1.0 00000100:00100000:3.0:1787826336.755897:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826336.755931:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346269568 00000100:00100000:3.0:1787826336.755935:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346269568 00000100:00100000:3.0:1787826336.755964:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30856 00000100:00100000:3.0:1787826336.755972:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346269568:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:3.0:1787826336.755994:0:35352:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1230 count 1 00010000:00010000:3.0:1787826336.756016:0:35352:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.756035:0:35352:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.756043:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4ce:0x0].0x0 since=8s v=(0 0) 00010000:00010000:3.0:1787826336.756047:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826336.756052:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.756080:0:35352:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.756100:0:35352:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 6 pid: 24218 timeout: 8248 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826336.756129:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346269696:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826336.756197:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3b0f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826336.756245:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346268800:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826336.756299:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346269440:12345-0@lo:1000:osp_up0-1.0 Request processed in 572us (836us total) trans 8589957432 rc 0/0 00000100:00100000:1.0:1787826336.756307:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23875 00000100:00100000:1.0:1787826336.756338:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826336.756574:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346269440:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826336.756588:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6857 rpc_version 6857 00000020:00100000:1.0:1787826336.756604:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826336.756610:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9ff100 x1874663346269952/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826336.756625:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346269952:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826336.756659:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826336.756723:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa300 going to sleep for 11 seconds 00000001:00080000:1.0:1787826336.756790:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976164 is committed 00000001:00080000:1.0:1787826336.756801:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976163 is committed 00000001:00080000:1.0:1787826336.756807:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976162 is committed 00000100:00100000:3.0:1787826336.757496:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346269952 00000100:00100000:3.0:1787826336.757504:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346269952 00000100:00100000:3.0:1787826336.757552:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23876 00000100:00100000:3.0:1787826336.757561:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346269952:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826336.758687:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346269952:12345-0@lo:1000:osp_up0-1.0 Request processed in 1123us (2023us total) trans 8589957433 rc 0/0 00000100:00100000:3.0:1787826336.758697:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23876 00000100:00100000:3.0:1787826336.758774:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346269952:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826336.758787:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6858 rpc_version 6858 00000020:00100000:3.0:1787826336.758806:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826336.758812:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784cc32680 x1874663346270208/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826336.758843:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346270208:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826336.758890:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826336.758912:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850751540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826336.758927:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346270208 00000100:00100000:3.0:1787826336.758931:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346270208 00000100:00100000:3.0:1787826336.758963:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23877 00000100:00100000:3.0:1787826336.758971:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346270208:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826336.759081:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346270208:12345-0@lo:1000:osp_up0-1.0 Request processed in 111us (190us total) trans 8589957434 rc 0/0 00000100:00100000:3.0:1787826336.759088:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23877 00000100:00100000:3.0:1787826336.759141:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346270208:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826336.759149:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6859 rpc_version 6859 00000020:00100000:3.0:1787826336.759266:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826336.759272:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977f29f80 x1874663346270464/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826336.759284:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29f80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346270464:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826336.759310:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826336.759326:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500b1840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826336.759345:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346270464 00000100:00100000:3.0:1787826336.759349:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346270464 00000100:00100000:3.0:1787826336.759373:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23878 00000100:00100000:3.0:1787826336.759379:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346270464:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826336.759454:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346270464:12345-0@lo:1000:osp_up0-1.0 Request processed in 74us (142us total) trans 8589957435 rc 0/0 00000100:00100000:3.0:1787826336.759461:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23878 00000100:00100000:3.0:1787826336.759507:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29f80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346270464:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826336.760466:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826336.760718:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:35352:1874663346269696:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826336.761253:0:35352:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826336.761407:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332635776 00000100:00100000:2.0:1787826336.761415:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332635776 00010000:00010000:2.0:1787826336.761522:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 7 pid: 24218 timeout: 8248 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826336.761563:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22000 00000100:00100000:2.0:1787826336.761574:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332635776:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826336.761588:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826336.761593:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 7 pid: 24218 timeout: 8248 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.761612:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 7 pid: 24218 timeout: 8248 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.761647:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae0b82f8/0xe21309aa81f4b322 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608016c5 expref: 7 pid: 24218 timeout: 8248 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.761660:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826336.761664:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826336.761669:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.761679:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.761715:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826336.761727:0:35352:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826336.761772:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332635776:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 198us (1309us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826336.761779:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22000 00010000:00010000:1.0:1787826336.761801:0:35352:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.761814:0:35352:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.763078:0:35352:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.763120:0:35352:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826336.763339:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346269568:12345-0@lo:6:osp-syn-0-1.0 Request processed in 7364us (7438us total) trans 4294973855 rc 0/0 00000100:00100000:1.0:1787826336.763349:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30856 00010000:00010000:2.0:1787826336.763514:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826336.763522:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.763533:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826336.763537:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.763553:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.763571:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.763580:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ff19e55/0xe21309aa81f4b686 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826336.763603:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826336.763605:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826336.763607:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1787826336.764186:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784f541500/2, rc 0, transno 4294973855 00000100:00100000:2.0:1787826336.764195:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346269568:0@lo:6:osp-syn-0-1.0 00000100:00080000:3.0:1787826336.765684:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1787826336.940532:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826337.964349:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826338.530599:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826338.530846:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332635904 00000100:00100000:2.0:1787826338.530860:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332635904 00000100:00100000:2.0:1787826338.530953:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68541 00000100:00100000:2.0:1787826338.530968:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225775:x1874663332635904:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826338.531003:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826338.531029:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b68d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017b3 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.531150:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.531168:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.531182:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826338.531244:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826338.531248:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826338.531270:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.531298:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4b68d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.531319:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.531330:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826338.531333:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826338.531338:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826338.531410:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225775:x1874663332635904:12345-192.168.203.56@tcp:101:cat.0 Request processed in 446us (823us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826338.531418:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68541 00000100:00080000:0.0:1787826338.532017:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826338.541474:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826338.541631:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332636032 00000100:00100000:3.0:1787826338.541645:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332636032 00000100:00100000:3.0:1787826338.541717:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68542 00000100:00100000:3.0:1787826338.541737:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225775:x1874663332636032:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826338.541764:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826338.541788:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b69b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017ba expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.541862:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.541877:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.541889:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.541905:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.541911:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.541917:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.541994:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b6b0 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542002:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b6b0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542008:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b6b0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826338.542030:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b6b0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542045:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542052:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542059:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542065:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542071:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826338.542076:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542095:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542130:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826338.542133:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826338.542139:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542145:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b6a2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542159:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542166:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542171:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542176:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826338.542178:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542184:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542190:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cce00) 00010000:00010000:3.0:1787826338.542193:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826338.542195:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542201:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6a9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542242:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b69b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017ba expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542255:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b6b0 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017ba expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.542262:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:3.0:1787826338.542264:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826338.542267:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826338.542333:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225775:x1874663332636032:12345-192.168.203.56@tcp:101:cat.0 Request processed in 606us (884us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826338.542341:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68542 00000100:00100000:2.0:1787826338.560425:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826338.560499:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332636160 00000100:00100000:2.0:1787826338.560512:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332636160 00000100:00100000:2.0:1787826338.560595:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68543 00000100:00100000:2.0:1787826338.560610:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:225775:x1874663332636160:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826338.560644:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826338.560674:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b6b7 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017c1 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.560775:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b6be lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.560792:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b6be lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.560810:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b6be lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826338.560934:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e54:0x0]: returning size 4096 00000004:00200000:2.0:1787826338.560940:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826338.560984:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b6b7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017c1 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.561004:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b6be lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017c1 expref: 6 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826338.561014:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826338.561018:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826338.561022:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826338.561239:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225775:x1874663332636160:12345-192.168.203.56@tcp:101:cat.0 Request processed in 482us (676us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826338.561247:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68543 00000100:00080000:2.0:1787826338.561279:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826338.570897:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826338.571014:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332636288 00000100:00100000:3.0:1787826338.571028:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332636288 00000100:00100000:3.0:1787826338.571138:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68544 00000100:00100000:3.0:1787826338.571152:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225775:x1874663332636288:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826338.571181:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826338.571209:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b6c5 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017c8 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.571297:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.571313:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.571327:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826338.571385:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x198:0x0]: returning size 4096 00000004:00200000:3.0:1787826338.571392:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826338.571433:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b6c5 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.571453:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826338.571481:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:3.0:1787826338.571485:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826338.571490:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826338.571563:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225775:x1874663332636288:12345-192.168.203.56@tcp:101:cat.0 Request processed in 414us (680us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826338.571570:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68544 00000100:00100000:0.0:1787826338.581591:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826338.584528:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332636416 00000100:00100000:1.0:1787826338.584551:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332636416 00000100:00100000:1.0:1787826338.584637:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68545 00000100:00100000:1.0:1787826338.584652:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225775:x1874663332636416:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826338.584680:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826338.584707:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b6d3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017cf expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826338.584780:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 3/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826338.584796:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826338.584810:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826338.584866:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x196:0x0]: returning size 4096 00000004:00200000:1.0:1787826338.584872:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826338.584916:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b6d3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826338.584935:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826338.584944:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826338.584948:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826338.584952:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826338.585041:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225775:x1874663332636416:12345-192.168.203.56@tcp:101:cat.0 Request processed in 391us (3463us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826338.585050:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68545 00000100:00100000:1.0:1787826338.595551:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826338.595724:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332636672 00000100:00100000:0.0:1787826338.595736:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332636672 00000100:00100000:0.0:1787826338.595807:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68546 00000100:00100000:0.0:1787826338.595819:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225775:x1874663332636672:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826338.595846:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826338.595869:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b6e1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017d6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826338.595955:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826338.595972:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826338.595984:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826338.595995:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 3/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826338.596003:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826338.596008:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:0.0:1787826338.596086:0:35098:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00010000:00010000:1.0:1787826338.988567:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826339.022641:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826339.022741:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332636800 00000100:00100000:2.0:1787826339.022755:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332636800 00000100:00100000:2.0:1787826339.022846:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68547 00000100:00100000:2.0:1787826339.022859:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225777:x1874663332636800:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826339.022892:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826339.022916:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b6f6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017dd expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.023016:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.023033:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.023046:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826339.023162:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826339.023167:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826339.023191:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.023222:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4b6f6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.023245:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.023255:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826339.023258:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826339.023263:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826339.023335:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225777:x1874663332636800:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 477us (709us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.023344:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68547 00000100:00080000:0.0:1787826339.023484:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826339.034569:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826339.035491:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332636928 00000100:00100000:2.0:1787826339.035508:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332636928 00000100:00100000:3.0:1787826339.035646:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68548 00000100:00100000:3.0:1787826339.035659:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225777:x1874663332636928:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826339.035698:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826339.035733:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b704 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017e4 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035829:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035846:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035859:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035877:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035884:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035889:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035969:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035989:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.035995:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826339.036042:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036059:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036065:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036072:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036079:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036086:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826339.036091:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036099:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036137:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826339.036140:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826339.036145:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036153:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b70b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036170:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036174:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036179:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036184:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826339.036186:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036191:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036197:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785ccd00) 00010000:00010000:3.0:1787826339.036199:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826339.036201:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036206:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b712 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036238:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b704 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036252:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.036260:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:3.0:1787826339.036263:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826339.036266:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826339.036667:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225777:x1874663332636928:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1006us (2110us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.036676:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68548 00000100:00100000:0.0:1787826339.041669:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826339.042306:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332637056 00000100:00100000:1.0:1787826339.042322:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332637056 00000100:00100000:1.0:1787826339.042422:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68549 00000100:00100000:1.0:1787826339.042437:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:225777:x1874663332637056:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826339.042478:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826339.042506:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b720 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017eb expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.042587:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b727 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.042605:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b727 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.042619:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b727 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826339.043585:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e54:0x0]: returning size 4096 00000004:00200000:1.0:1787826339.043593:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826339.043648:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b720 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017eb expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.043671:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b727 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017eb expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.043686:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:1.0:1787826339.043689:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826339.043694:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826339.043890:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225777:x1874663332637056:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1455us (2234us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.043898:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68549 00000100:00080000:0.0:1787826339.046740:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826339.047264:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826339.047417:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332637184 00000100:00100000:2.0:1787826339.047430:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332637184 00000100:00100000:2.0:1787826339.047512:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68550 00000100:00100000:2.0:1787826339.047525:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225777:x1874663332637184:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826339.047553:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826339.047578:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4b72e lrc: 2/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017f2 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.047661:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.047678:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.047692:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.047710:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.047718:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.047737:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048065:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b743 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048075:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b743 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048083:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b743 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826339.048177:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e55:0x0]: returning size 6 00000004:00200000:2.0:1787826339.048182:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826339.048205:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b743 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048219:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b743 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048229:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048236:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048244:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048251:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048258:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826339.048263:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048272:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048285:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:2.0:1787826339.048289:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826339.048294:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048300:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4b735 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048318:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048323:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048329:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048335:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826339.048337:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048344:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048351:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x4ec94fd6 (ffff9f7967ac8600) 00010000:00010000:2.0:1787826339.048353:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826339.048356:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048362:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef600/0xe21309aa81f4b73c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048863:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4b72e lrc: 0/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017f2 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048885:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b743 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017f2 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.048893:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f7967ac8d00) 00010000:00010000:2.0:1787826339.048896:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826339.048898:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826339.049017:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225777:x1874663332637184:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1493us (1761us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.049026:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68550 00000100:00100000:2.0:1787826339.062382:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826339.063008:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332637312 00000100:00100000:3.0:1787826339.063024:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332637312 00000100:00100000:3.0:1787826339.063134:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68551 00000100:00100000:3.0:1787826339.063148:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225777:x1874663332637312:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826339.063176:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826339.063201:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b74a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608017f9 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063285:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063302:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063315:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063328:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063336:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063344:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063468:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b75f lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063477:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b75f lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063485:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b75f lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826339.063537:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbe:0x0]: returning size 6 00000004:00200000:3.0:1787826339.063542:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826339.063566:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b75f lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063580:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b75f lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063590:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063597:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063603:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063610:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063621:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826339.063625:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063633:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063647:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:3.0:1787826339.063650:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826339.063656:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063662:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b751 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063680:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063687:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063692:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063697:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826339.063700:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063705:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.063716:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f7961f9a600) 00010000:00010000:3.0:1787826339.063719:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826339.063721:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.065037:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4b758 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.065101:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4b74a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017f9 expref: 13 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.065139:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b75f lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017f9 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.065149:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826339.065152:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826339.065156:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826339.065251:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225777:x1874663332637312:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2103us (2880us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.065261:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68551 00000100:00100000:0.0:1787826339.074014:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826339.075338:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332637440 00000100:00100000:1.0:1787826339.075355:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332637440 00000100:00100000:1.0:1787826339.075465:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11149 00000100:00100000:1.0:1787826339.075479:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332637440:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:1.0:1787826339.075499:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826339.075530:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4b743 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017f2 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.075559:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f7967ac8d00) 00010000:00010000:1.0:1787826339.075572:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826339.075651:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332637440:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 175us (1652us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.075660:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11149 00000100:00100000:2.0:1787826339.087387:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826339.087514:0:6541:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332637696 00000100:00100000:1.0:1787826339.087527:0:6541:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332637696 00000100:00100000:1.0:1787826339.087617:0:6541:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1233 00000100:00100000:1.0:1787826339.087630:0:6541:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225777:x1874663332637696:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:1.0:1787826339.087695:0:6541:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826339.087728:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4b75f lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017f9 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.087769:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:1.0:1787826339.087783:0:6541:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826339.087879:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346270592:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.087958:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.088039:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826339.088247:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346270592 00000100:00100000:0.0:1787826339.088254:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346270592 00000100:00100000:0.0:1787826339.088299:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23879 00000100:00100000:0.0:1787826339.088308:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346270592:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826339.088395:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346270592:12345-0@lo:1000:mdt_io00_001.0 Request processed in 87us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826339.088402:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23879 00000100:00100000:1.0:1787826339.089160:0:6541:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985428 gen 2 00000004:00080000:1.0:1787826339.089170:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985427 committed transno 4294985428 00000020:00080000:1.0:1787826339.089179:0:6541:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f784dedd180 refcount 2 committed 0 result 0 batchid 4294979085 00000020:00080000:1.0:1787826339.089183:0:6541:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784a925300 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784dedd6c0 00000020:00080000:1.0:1787826339.089187:0:6541:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].941 00000020:00080000:1.0:1787826339.089189:0:6541:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed35d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7847b18c00 00000020:00080000:1.0:1787826339.089193:0:6541:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].938 00000004:00080000:1.0:1787826339.089221:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985428 committed transno 4294985428 00000100:00100000:1.0:1787826339.089243:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346270592:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.089327:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346270720:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.089368:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.089419:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000020:00080000:0.0:1787826339.089542:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979085 committed batchid 4294979084 00000020:00080000:0.0:1787826339.089621:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979084 00000100:00100000:3.0:1787826339.089716:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346270720 00000100:00100000:3.0:1787826339.089734:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346270720 00000100:00100000:3.0:1787826339.089799:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23880 00000100:00100000:3.0:1787826339.089810:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346270720:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.089913:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346270720:12345-0@lo:1000:mdt_io00_001.0 Request processed in 105us (544us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.089922:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23880 00000100:00100000:1.0:1787826339.090587:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346270720:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826339.090633:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x198:0x0] 00000100:00100000:1.0:1787826339.090680:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346270848:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.090736:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.090774:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826339.090931:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346270848 00000100:00100000:0.0:1787826339.090939:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346270848 00000100:00100000:0.0:1787826339.090996:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23881 00000100:00100000:0.0:1787826339.091006:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346270848:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826339.091099:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346270848:12345-0@lo:1000:mdt_io00_001.0 Request processed in 94us (375us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826339.091132:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23881 00000100:00100000:2.0:1787826339.091270:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346270848:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826339.091295:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e54:0x0] 00000100:00100000:2.0:1787826339.091333:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346270976:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826339.091365:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826339.091403:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.091601:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346270976 00000100:00100000:3.0:1787826339.091607:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346270976 00000100:00100000:3.0:1787826339.091648:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23882 00000100:00100000:3.0:1787826339.091657:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346270976:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.091743:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346270976:12345-0@lo:1000:mdt_io00_001.0 Request processed in 87us (376us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.091756:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23882 00000100:00100000:2.0:1787826339.092007:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346270976:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826339.092058:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x198:0x0] 00000100:00100000:2.0:1787826339.092088:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346271104:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826339.092152:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826339.092170:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78457856c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.092708:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346271104 00000100:00100000:3.0:1787826339.092716:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346271104 00000100:00100000:3.0:1787826339.092780:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23883 00000100:00100000:3.0:1787826339.092790:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346271104:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.092891:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346271104:12345-0@lo:1000:mdt_io00_001.0 Request processed in 101us (737us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.092898:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23883 00000100:00100000:2.0:1787826339.093292:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346271104:0@lo:1000:mdt_io00_001.0 00000004:00020000:2.0:1787826339.093340:0:6541:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:2.0:1787826339.093417:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.093430:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.093442:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826339.093473:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346271232:0@lo:101:mdt_io00_001.0 00000100:00100000:2.0:1787826339.093520:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826339.093557:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785f00 going to sleep for 11 seconds 00000004:00020000:0.0:1787826339.093929:0:35098:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4502 00010000:00010000:0.0:1787826339.094020:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094038:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094051:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826339.094066:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbe:0x0]: returning size 6 00010000:00010000:0.0:1787826339.094222:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094232:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094238:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094245:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094251:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826339.094256:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094264:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094279:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:0.0:1787826339.094283:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826339.094288:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094295:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b6e8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094314:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094321:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094327:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094332:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826339.094335:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094341:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094348:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f7961f9a600) 00010000:00010000:0.0:1787826339.094351:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826339.094354:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 2/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094360:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b6ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094422:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b6e1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094438:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.094453:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f7845784900) 00010000:00010000:0.0:1787826339.094466:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826339.094469:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826339.094539:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225775:x1874663332636672:12345-192.168.203.56@tcp:101:cat.0 Request processed in 498721us (499000us total) trans 8589957436 rc 0/0 00000100:00100000:0.0:1787826339.094548:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68546 00000100:00100000:3.0:1787826339.098825:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346271232 00000100:00100000:3.0:1787826339.098847:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346271232 00000100:00100000:3.0:1787826339.099044:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23884 00000100:00100000:3.0:1787826339.099057:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346271232:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826339.099086:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826339.099137:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 2/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b766 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.099181:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 6 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.099193:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 4/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b766 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.099242:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 6 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.099258:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 6 pid: 11629 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826339.099273:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6533:1874663346271360:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826339.099387:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850751540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.099480:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826339.101368:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332637824 00000100:00100000:0.0:1787826339.101386:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332637824 00000100:00100000:0.0:1787826339.101476:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30857 00000100:00100000:0.0:1787826339.101491:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332637824:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826339.101525:0:20167:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826339.101547:0:20167:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b782 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801800 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826339.101616:0:20167:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346271488:192.168.203.56@tcp:106: 00000100:00100000:0.0:1787826339.101715:0:20167:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247cc0 going to sleep for 11 seconds 00000100:00080000:3.0:1787826339.102147:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826339.106407:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6533:1874663346271360:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826339.106467:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 3/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b766 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.106483:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:1.0:1787826339.106486:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826339.106492:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 3/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b766 expref: 6 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.106516:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68800, rc 0) 00000100:00100000:1.0:1787826339.106591:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6541:x1874663346271232:12345-0@lo:101:mdt_io00_001.0 Request processed in 7537us (13069us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.106601:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23884 00000100:00100000:2.0:1787826339.106717:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346271232:0@lo:101:mdt_io00_001.0 00010000:00010000:2.0:1787826339.106739:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.106750:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826339.106855:0:20167:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000401:0x521:0x0] updating lvb size: 0 -> 6 00002000:00010000:0.0:1787826339.106859:0:20167:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000401:0x521:0x0] updating lvb mtime: 0 -> 1787826333 00002000:00010000:0.0:1787826339.106862:0:20167:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000401:0x521:0x0] updating lvb atime: 0 -> 1787826333 00002000:00010000:0.0:1787826339.106864:0:20167:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000401:0x521:0x0] updating lvb ctime: 0 -> 1787826333 00002000:00010000:0.0:1787826339.106867:0:20167:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000401:0x521:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:0.0:1787826339.106888:0:20167:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346271488:192.168.203.56@tcp:106: 00010000:00010000:0.0:1787826339.106919:0:20167:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b782 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801800 expref: 6 pid: 20167 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.106948:0:20167:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e08490e4/0xe21309aa81f4b782 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801800 expref: 6 pid: 20167 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.106961:0:20167:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826339.107022:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332637824:12345-192.168.203.56@tcp:101:cat.0 Request processed in 5535us (7542us total) trans 0 rc 301/301 00000100:00100000:0.0:1787826339.107030:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30857 00000100:00100000:2.0:1787826339.115223:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826339.115325:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332637952 00000100:00100000:1.0:1787826339.115338:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332637952 00010000:00010000:1.0:1787826339.115405:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 7 pid: 11629 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.115467:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22001 00000100:00100000:1.0:1787826339.115480:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332637952:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826339.115500:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826339.115519:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 7 pid: 11629 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.115535:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 7 pid: 11629 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.115569:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4b6cc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017c8 expref: 7 pid: 11629 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.115590:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:1.0:1787826339.115599:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 2/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b766 expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.115611:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b766 expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.115621:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b766 expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.115663:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b766 expref: 5 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.115680:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6511:1874663346271616:0@lo:105: 00000100:00100000:1.0:1787826339.115740:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.115800:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.116084:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346271616 00000100:00100000:3.0:1787826339.116092:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346271616 00000100:00100000:3.0:1787826339.116171:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7892 00000100:00100000:3.0:1787826339.116180:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6511:x1874663346271616:12345-0@lo:105: 00010000:00010000:3.0:1787826339.116194:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.116205:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.116214:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.116223:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.116234:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.116243:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.116273:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ee200) 00000100:00100000:3.0:1787826339.116326:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6511:x1874663346271616:12345-0@lo:105: Request processed in 147us (584us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.116334:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7892 00010000:00010000:0.0:1787826339.116486:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.116497:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826339.116538:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x198:0x0] 00010000:00010000:0.0:1787826339.116587:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.116597:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.116607:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826339.116628:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346271744:0@lo:101:mdt_io00_001.0 00000100:00100000:0.0:1787826339.116664:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826339.116696:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.117130:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346271744 00000100:00100000:3.0:1787826339.117138:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346271744 00000100:00100000:3.0:1787826339.117195:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23885 00000100:00100000:3.0:1787826339.117205:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6541:x1874663346271744:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826339.117225:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826339.117245:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b790 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b789 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.117268:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b790 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b789 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.117282:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b790 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b789 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.117290:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x4ec94fd6 (ffff9f79785ccf00) 00010000:00010000:3.0:1787826339.117294:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826339.117297:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826339.117347:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346271744:12345-0@lo:101:mdt_io00_001.0 Request processed in 144us (683us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.117354:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23885 00000100:00100000:2.0:1787826339.117463:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346271744:0@lo:101:mdt_io00_001.0 00010000:00010000:2.0:1787826339.117483:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b790 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.117494:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b790 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826339.117522:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x198:0x0] 00010000:00010000:2.0:1787826339.117543:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.117562:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.117570:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.117601:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.117617:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 10 pid: 17813 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826339.117630:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346271872:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826339.117713:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.118070:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6511:1874663346271616:0@lo:105: 00010000:00010000:1.0:1787826339.118088:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826339.118141:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332637952:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2663us (2932us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.118151:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22001 00000100:00100000:0.0:1787826339.121729:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346271872:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826339.121783:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826339.123393:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826339.123652:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332638080 00000100:00100000:1.0:1787826339.123665:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332638080 00010000:00010000:1.0:1787826339.123719:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 11 pid: 17813 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.123783:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22002 00000100:00100000:1.0:1787826339.123796:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7953caac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332638080:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826339.123815:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826339.123824:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 11 pid: 17813 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.123839:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 11 pid: 17813 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.123865:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b6da lrc: 0/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017cf expref: 11 pid: 17813 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.123884:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826339.123890:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.123900:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.123908:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.123941:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826339.124012:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7953caac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332638080:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 218us (639us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.124020:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22002 00010000:00010000:0.0:1787826339.124589:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.124601:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.124626:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.124637:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.124646:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826339.124719:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272000:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826339.124800:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826339.124836:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.125298:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346272000 00000100:00100000:1.0:1787826339.125308:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346272000 00000100:00100000:1.0:1787826339.125364:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23886 00000100:00100000:1.0:1787826339.125386:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272000:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.125506:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272000:12345-0@lo:1000:mdt_io00_001.0 Request processed in 133us (705us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.125515:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23886 00000100:00100000:0.0:1787826339.126864:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272000:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826339.126965:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272128:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826339.127036:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826339.127070:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.127397:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346272128 00000100:00100000:3.0:1787826339.127406:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346272128 00000100:00100000:3.0:1787826339.127476:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23887 00000100:00100000:3.0:1787826339.127488:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272128:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.127913:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272128:12345-0@lo:1000:mdt_io00_001.0 Request processed in 425us (875us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.127921:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23887 00000100:00100000:2.0:1787826339.129521:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826339.129958:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332638208 00000100:00100000:0.0:1787826339.129970:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332638208 00000100:00100000:0.0:1787826339.130050:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30858 00000100:00100000:0.0:1787826339.130063:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:225775:x1874663332638208:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826339.130085:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826339.130133:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801807 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.130184:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 5 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.130195:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x521:0x0].0x0 since=5s v=(0 0) 00010000:00010000:0.0:1787826339.130200:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826339.130207:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.130247:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 5 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.130262:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 5 pid: 20167 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826339.130276:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346272256:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826339.130391:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.131342:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272128:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.131450:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272384:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.131517:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.131559:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.131631:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346272384 00000100:00100000:3.0:1787826339.131639:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346272384 00000100:00100000:3.0:1787826339.131688:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23888 00000100:00100000:3.0:1787826339.131698:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272384:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.131876:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272384:12345-0@lo:1000:mdt_io00_001.0 Request processed in 178us (357us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826339.131884:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23888 00000100:00100000:1.0:1787826339.132292:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272384:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.132480:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272512:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.132516:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.132547:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.132673:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346272512 00000100:00100000:3.0:1787826339.132680:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346272512 00000100:00100000:3.0:1787826339.132736:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23889 00000100:00100000:3.0:1787826339.132745:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272512:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.132836:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272512:12345-0@lo:1000:mdt_io00_001.0 Request processed in 91us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.132843:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23889 00000100:00100000:2.0:1787826339.132915:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272512:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826339.133009:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272640:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826339.133038:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826339.133063:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.133355:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346272640 00000100:00100000:3.0:1787826339.133361:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346272640 00000100:00100000:3.0:1787826339.133413:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23890 00000100:00100000:3.0:1787826339.133422:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272640:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.133500:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346272640:12345-0@lo:1000:mdt_io00_001.0 Request processed in 80us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826339.133508:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23890 00000100:00100000:2.0:1787826339.133702:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346272640:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826339.133742:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e54:0x0] 00010000:00010000:2.0:1787826339.133837:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.133858:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.133868:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.133895:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 9 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.133907:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 9 pid: 35098 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826339.133918:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346272768:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826339.134093:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78457856c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826339.139047:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826339.139090:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826339.139176:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332638464 00000100:00100000:2.0:1787826339.139188:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332638464 00010000:00010000:2.0:1787826339.139239:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 10 pid: 35098 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826339.139275:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332638336 00000100:00100000:2.0:1787826339.139279:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332638336 00010000:00010000:2.0:1787826339.139308:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 6 pid: 20167 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826339.139334:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22003 00000100:00100000:2.0:1787826339.139345:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332638464:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826339.139363:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826339.139648:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346272768:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826339.139681:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.140002:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22004 00000100:00100000:1.0:1787826339.140018:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332638336:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826339.140030:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826339.140059:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 6 pid: 20167 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.140075:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 6 pid: 20167 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.140129:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826339.140136:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826339.140147:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.140171:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50080000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.140217:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.140233:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:125406:1874663346272896:192.168.203.56@tcp:105: 00010000:00010000:0.0:1787826339.140426:0:17826:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4b655 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080179e expref: 6 pid: 20167 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826339.140443:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:17826:1874663346272256:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826339.140476:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.140488:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:1.0:1787826339.140548:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aab40 going to sleep for 11 seconds 00010000:00010000:2.0:1787826339.140658:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 10 pid: 35098 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.140671:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 10 pid: 35098 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.140688:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b76d lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017d6 expref: 10 pid: 35098 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.140702:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f7845784900) 00010000:00010000:2.0:1787826339.140707:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.140717:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.140731:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.140756:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826339.140807:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.140815:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.140870:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.140877:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.140887:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.140904:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273024:0@lo:101:mdt_io00_001.0 00000100:00100000:1.0:1787826339.140950:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.140990:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346273024 00000100:00100000:1.0:1787826339.140996:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346273024 00000100:00100000:1.0:1787826339.141041:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23891 00000100:00100000:1.0:1787826339.141051:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346273024:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826339.141070:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826339.141088:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b7ba lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b7b3 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.141126:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b7ba lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b7b3 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.141137:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b7ba lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b7b3 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.141144:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:1.0:1787826339.141147:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787826339.141150:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826339.141173:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273024:12345-0@lo:101:mdt_io00_001.0 Request processed in 124us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.141179:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23891 00000100:00100000:1.0:1787826339.141202:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.141218:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f28380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273024:0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826339.141232:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b7ba expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.141242:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b7ba expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826339.141267:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e55:0x0] 00000100:00100000:1.0:1787826339.141316:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273152:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.141342:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.141358:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.142316:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346273152 00000100:00100000:1.0:1787826339.142330:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346273152 00000100:00100000:1.0:1787826339.142400:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23892 00000100:00100000:1.0:1787826339.142411:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273152:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.142552:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273152:12345-0@lo:1000:mdt_io00_001.0 Request processed in 143us (1208us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.142560:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23892 00000100:00100000:1.0:1787826339.142617:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273152:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826339.142657:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x198:0x0] 00000100:00100000:1.0:1787826339.142708:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273280:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.142958:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.143002:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346273280 00000100:00100000:1.0:1787826339.143008:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346273280 00000100:00100000:1.0:1787826339.143036:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23893 00000100:00100000:1.0:1787826339.143044:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273280:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.143130:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273280:12345-0@lo:1000:mdt_io00_001.0 Request processed in 61us (147us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.143137:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23893 00000100:00100000:1.0:1787826339.143275:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453cef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.143293:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273280:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826339.143316:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e54:0x0] 00010000:00010000:1.0:1787826339.143360:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.143451:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.143476:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.143531:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273408:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.143561:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.143590:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346273408 00000100:00100000:1.0:1787826339.143595:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346273408 00000100:00100000:1.0:1787826339.143623:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23894 00000100:00100000:1.0:1787826339.143630:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273408:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.143686:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273408:12345-0@lo:1000:mdt_io00_001.0 Request processed in 56us (124us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.143693:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23894 00000100:00100000:1.0:1787826339.143713:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.143885:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273408:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826339.144153:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225775:x1874663332638208:12345-192.168.203.56@tcp:101:cat.0 Request processed in 14087us (14637us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826339.144173:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30858 00000004:00080000:1.0:1787826339.144287:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e54:0x0] 00000100:00100000:1.0:1787826339.144319:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273536:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.144354:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.144449:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453cef00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826339.145988:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332638464:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 6624us (6936us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.146005:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22003 00000100:00080000:2.0:1787826339.146062:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826339.146089:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346273536 00000100:00100000:2.0:1787826339.146099:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346273536 00000100:00100000:2.0:1787826339.146178:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23895 00000100:00100000:2.0:1787826339.146194:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273536:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826339.146330:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273536:12345-0@lo:1000:mdt_io00_001.0 Request processed in 142us (1974us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.146338:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23895 00000100:00100000:1.0:1787826339.147412:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273536:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.147522:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273664:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.147583:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.147634:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453cef00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826339.148772:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346273664 00000100:00100000:2.0:1787826339.148790:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346273664 00000100:00100000:2.0:1787826339.148861:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23896 00000100:00100000:2.0:1787826339.148874:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273664:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826339.149038:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273664:12345-0@lo:1000:mdt_io00_001.0 Request processed in 164us (1452us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.149047:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23896 00000100:00100000:1.0:1787826339.149364:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273664:0@lo:1000:mdt_io00_001.0 00000004:00080000:1.0:1787826339.149420:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x198:0x0] 00000100:00100000:1.0:1787826339.149488:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273792:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.149528:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.149556:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453cef00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826339.149734:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346273792 00000100:00100000:2.0:1787826339.149741:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346273792 00000100:00100000:2.0:1787826339.149793:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23897 00000100:00100000:2.0:1787826339.149802:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273792:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826339.149891:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273792:12345-0@lo:1000:mdt_io00_001.0 Request processed in 90us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.149899:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23897 00000100:00100000:3.0:1787826339.149988:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273792:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826339.150012:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e54:0x0] 00000100:00100000:3.0:1787826339.150069:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273920:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.150102:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826339.150151:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850751540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.150500:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346273920 00000100:00100000:1.0:1787826339.150506:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346273920 00000100:00100000:1.0:1787826339.150550:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23898 00000100:00100000:1.0:1787826339.150558:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273920:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.150628:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346273920:12345-0@lo:1000:mdt_io00_001.0 Request processed in 71us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.150636:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23898 00000100:00100000:3.0:1787826339.150720:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346273920:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.150772:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346274048:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.150797:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826339.150822:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850751600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826339.150923:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346274048 00000100:00100000:1.0:1787826339.150928:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346274048 00000100:00100000:1.0:1787826339.150961:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23899 00000100:00100000:1.0:1787826339.150970:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346274048:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826339.151045:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346274048:12345-0@lo:1000:mdt_io00_001.0 Request processed in 76us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.151051:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23899 00000100:00100000:3.0:1787826339.151484:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346274048:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.151574:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346274176:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826339.151605:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826339.151632:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850751540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826339.151679:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346274176 00000100:00100000:2.0:1787826339.151686:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346274176 00000100:00100000:2.0:1787826339.151731:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23900 00000100:00100000:2.0:1787826339.151739:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346274176:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826339.151815:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346274176:12345-0@lo:1000:mdt_io00_001.0 Request processed in 76us (209us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.151821:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23900 00000100:00100000:3.0:1787826339.152327:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c95c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346274176:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826339.152826:0:6541:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1dbe:0x0] is inserted into orphan list, open count = 1 00000040:00080000:3.0:1787826339.152898:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].939, 1240 off884448 00000040:00080000:3.0:1787826339.152967:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].943, 1320 off984360 00000004:00080000:3.0:1787826339.152987:0:6541:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784a3b0f00 with version 1832 00000004:00080000:0.0:1787826339.153407:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1832 rpc_version 1832 00000020:00100000:0.0:1787826339.153443:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826339.153464:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7850ad5880 x1874663346274432/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3432/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826339.153490:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346274432:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826339.153559:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826339.153592:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7849247cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826339.153935:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346274432 00000100:00100000:3.0:1787826339.153944:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346274432 00000100:00100000:3.0:1787826339.154268:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23901 00000100:00100000:3.0:1787826339.154281:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8a050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346274432:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826339.154840:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8a050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346274432:12345-0@lo:1000:osp_up1-0.0 Request processed in 560us (1279us total) trans 4294985438 rc 0/0 00000100:00100000:3.0:1787826339.154850:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23901 00000100:00100000:0.0:1787826339.155636:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346274432:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787826339.155827:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155841:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155849:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155857:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826339.155863:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155871:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155889:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f7845784900) 00010000:00010000:1.0:1787826339.155892:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826339.155898:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155905:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4b7ac lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826339.155931:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e55:0x0] 00010000:00010000:1.0:1787826339.155938:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b7ba expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155947:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b7ba expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155961:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155967:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155987:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.155997:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826339.156000:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156007:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156014:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826339.156016:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826339.156020:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156035:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4b797 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826339.156049:0:6541:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c9a7dd3d reply state 00000000d2077935 transno 8589957437 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156062:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156070:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f7961f9a600) 00010000:00010000:1.0:1787826339.156072:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826339.156076:0:6541:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:1.0:1787826339.156080:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x198:0x0] 00010000:00010000:1.0:1787826339.156084:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156092:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156140:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156342:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826339.156350:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x198:0x0] 00010000:00010000:1.0:1787826339.156357:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b790 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156364:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b790 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156381:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156387:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156393:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156398:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826339.156401:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156407:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156416:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7967ac1200) 00010000:00010000:1.0:1787826339.156418:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826339.156422:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.156428:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4b7c1 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.156556:0:6541:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225777:x1874663332637696:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 68924us (69194us total) trans 8589957437 rc 0/0 00000100:00100000:1.0:1787826339.156566:0:6541:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1233 00010000:00010000:0.0:1787826339.157185:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826339.157196:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.157206:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826339.157213:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826339.157221:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.157270:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.157281:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826339.157319:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.157326:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f78490ee200/0xe21309aa81f4b766 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b77b expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.157348:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826339.157350:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826339.157352:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826339.159554:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826339.159994:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346274560:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826339.160062:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.160142:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332638592 00000100:00100000:1.0:1787826339.160151:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332638592 00000100:00100000:1.0:1787826339.160217:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68552 00000100:00100000:1.0:1787826339.160227:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225775:x1874663332638592:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826339.160254:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826339.160280:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4b7c8 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080180e expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.160345:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4b7c8 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080180e expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.160360:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4b7c8 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080180e expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.160391:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:1.0:1787826339.160395:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787826339.160399:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826339.160447:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225775:x1874663332638592:12345-192.168.203.56@tcp:101:cat.0 Request processed in 221us (906us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.160464:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68552 00000100:00100000:1.0:1787826339.160515:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:125406:1874663346272896:192.168.203.56@tcp:105: 00010000:00010000:1.0:1787826339.160535:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826339.160554:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332638336:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 20544us (21463us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.160562:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22004 00000100:00100000:1.0:1787826339.160578:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346274560 00000100:00100000:1.0:1787826339.160580:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346274560 00000100:00100000:1.0:1787826339.160608:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11150 00000100:00100000:1.0:1787826339.160614:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346274560:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826339.160623:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826339.160642:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4b77b lrc: 0/0,0 mode: --/CW res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b766 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.160658:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:1.0:1787826339.160667:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826339.160693:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346274560:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (632us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.160699:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11150 00000100:00100000:1.0:1787826339.160726:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346274560:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826339.168631:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826339.168886:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332638720 00000100:00100000:2.0:1787826339.168900:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332638720 00002000:00100000:2.0:1787826339.168968:0:8448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x521:0x0] (0->4095) req@ffff9f7842c95500 x1874663332638720/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:695/0 lens 488/0 e 0 to 0 dl 1787826350 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826339.168997:0:8448:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7842c95500 with x1874663332638720 ext(0->4095) 00002000:00010000:2.0:1787826339.169004:0:8448:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7842c95500 00000100:00100000:2.0:1787826339.169044:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27658 00000100:00100000:2.0:1787826339.169056:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332638720:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:2.0:1787826339.169476:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332638720:12345-192.168.203.56@tcp:3:cat.0 Request processed in 400us (839us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787826339.169491:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27658 00002000:00100000:2.0:1787826339.169504:0:8448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x521:0x0] (0->4095) req@ffff9f7842c95500 x1874663332638720/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:695/0 lens 488/4536 e 0 to 0 dl 1787826350 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826339.169516:0:8448:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7842c95500 with x1874663332638720 ext(0->4095) 00002000:00010000:2.0:1787826339.169522:0:8448:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7842c95500 00000100:00100000:0.0:1787826339.181922:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826339.182601:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332638976 00000100:00100000:3.0:1787826339.182616:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332638976 00000100:00100000:3.0:1787826339.182697:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9368 00000100:00100000:3.0:1787826339.182712:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943032850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225775:x1874663332638976:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:3.0:1787826339.183362:0:17601:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1dbe:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1787826339.183590:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].215, 64 off21952 00000004:00080000:3.0:1787826339.183604:0:17601:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8151395497640ns, next at 8152395494190ns, rc = 0 00010000:00010000:3.0:1787826339.183677:0:17601:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.183700:0:17601:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.183714:0:17601:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.183795:0:17601:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.183804:0:17601:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.183810:0:17601:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.183892:0:17601:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826339.184039:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943032850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225775:x1874663332638976:12345-192.168.203.56@tcp:35:cat.0 Request processed in 1327us (2127us total) trans 8589957438 rc 0/0 00000100:00100000:3.0:1787826339.184050:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9368 00010000:00010000:1.0:1787826339.185323:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826339.185337:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.185349:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69a00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826339.185356:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.185366:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.185395:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:1.0:1787826339.185399:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826339.185404:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.185411:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4b7cf lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dbe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 17601 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.185430:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826339.185432:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826339.185434:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826339.778101:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b790 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826339.778140:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b790 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.778187:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b790 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.778194:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826339.778234:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43200/0xe21309aa81f4b789 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b790 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.778491:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b7ba expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826339.778505:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b7ba expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.778524:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b7ba expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.778530:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826339.778537:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfa00/0xe21309aa81f4b7b3 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b7ba expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826339.778551:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957438 is committed 00000004:00080000:0.0:1787826339.778558:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 214 committed 00000001:00080000:0.0:1787826339.778592:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957437 is committed 00000020:00080000:0.0:1787826339.778608:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979085 batchid updated 00000001:00080000:0.0:1787826339.778618:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957435 is committed 00000100:00100000:2.0:1787826339.778944:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346274688:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826339.779020:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826339.779064:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787826339.779071:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346274816:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826339.779092:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826339.779141:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346274688 00000100:00100000:2.0:1787826339.779148:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346274688 00000100:00100000:2.0:1787826339.779195:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346274816 00000100:00100000:2.0:1787826339.779198:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346274816 00000100:00100000:2.0:1787826339.779222:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11151 00000100:00100000:2.0:1787826339.779231:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346274688:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826339.779244:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826339.779437:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11152 00000100:00100000:1.0:1787826339.779473:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346274816:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826339.779494:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826339.779525:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.779540:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.779541:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b7ba lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b7b3 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.779580:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826339.779583:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:0.0:1787826339.779593:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826339.779599:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.779606:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.779620:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f7961f9a600) 00010000:00010000:0.0:1787826339.779623:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826339.779627:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.779632:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b640 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.779742:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4b790 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b789 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.779757:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x4ec94fd6 (ffff9f79785ccf00) 00010000:00010000:2.0:1787826339.779766:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826339.779812:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346274688:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 581us (791us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.779819:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11151 00000100:00100000:2.0:1787826339.779977:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346274688:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826339.780023:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826339.780024:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.780035:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.780042:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.780048:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826339.780052:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.780059:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.780070:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f7961f9a600) 00010000:00010000:2.0:1787826339.780074:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826339.780077:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.780085:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b79e lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826339.780145:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346274816:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 691us (1047us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826339.780154:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11152 00000100:00100000:1.0:1787826339.780192:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826339.780208:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346274944:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1787826339.780249:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826339.780317:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346274816:0@lo:103:jbd2/dm-0-8.0 00000020:00080000:0.0:1787826339.781791:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979085 00000100:00100000:2.0:1787826339.781964:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346274944 00000100:00100000:2.0:1787826339.781973:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346274944 00000100:00100000:2.0:1787826339.782033:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30859 00000100:00100000:2.0:1787826339.782042:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346274944:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1787826339.782070:0:8444:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1313 count 1 00010000:00010000:2.0:1787826339.782095:0:8444:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.782129:0:8444:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.782138:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826339.782142:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826339.782146:0:8444:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.782177:0:8444:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.782192:0:8444:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 5 pid: 17826 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826339.782206:0:8444:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8444:1874663346275072:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826339.782304:0:8444:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78457853c0 going to sleep for 11 seconds 00000100:00080000:2.0:1787826339.782324:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:0.0:1787826339.782416:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7849247b40 with version 1833 00000020:00080000:0.0:1787826339.782436:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979085 cancel update log [0x240000404:0x2:0x0].941: rc = 0 00000020:00080000:0.0:1787826339.782536:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979085 cancel update log [0x200000bd0:0x2:0x0].938: rc = 0 00000020:00080000:0.0:1787826339.782548:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979085 00000004:00080000:2.0:1787826339.784652:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1833 rpc_version 1833 00000020:00100000:2.0:1787826339.784696:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826339.784718:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9e9c00 x1874663346275328/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826339.787152:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346275328:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826339.787232:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826339.787278:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826339.787529:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346275328 00000100:00100000:0.0:1787826339.787538:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346275328 00000100:00100000:0.0:1787826339.787670:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23902 00000100:00100000:0.0:1787826339.787681:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346275328:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826339.787881:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346275328:12345-0@lo:1000:osp_up1-0.0 Request processed in 201us (648us total) trans 4294985439 rc 0/0 00000100:00100000:0.0:1787826339.787888:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23902 00000100:00100000:2.0:1787826339.787970:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346275328:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826339.790337:0:8444:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8444:1874663346275072:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826339.790641:0:8444:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826339.790712:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826339.791015:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332639104 00000100:00100000:2.0:1787826339.791023:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332639104 00010000:00010000:2.0:1787826339.791059:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826339.791097:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22005 00000100:00100000:2.0:1787826339.791149:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332639104:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826339.791164:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826339.791170:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.791183:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.791207:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c5bf838/0xe21309aa81f4b7a5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x521:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801807 expref: 6 pid: 17826 timeout: 8251 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.791222:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=1s v=(1 0) 00010000:00010000:2.0:1787826339.791227:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x521:0x0].0x0 since=0s v=(0 1) 00010000:00010000:2.0:1787826339.791233:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.791242:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826339.791279:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826339.791327:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332639104:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 222us (618us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826339.791334:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22005 00010000:00010000:3.0:1787826339.796473:0:8444:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.796495:0:8444:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.796506:0:8444:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.796513:0:8444:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826339.796544:0:8444:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826339.796780:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346274944:12345-0@lo:6:osp-syn-0-0.0 Request processed in 14734us (16526us total) trans 4294973856 rc 0/0 00000100:00100000:3.0:1787826339.796789:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30859 00010000:00010000:0.0:1787826339.799633:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826339.799650:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.799664:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efc00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826339.799672:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.799684:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.799707:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.799714:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010d0e8fb/0xe21309aa81f4b7d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x521:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826339.799747:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826339.799749:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826339.799751:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1787826339.800075:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7850ad4e00/2, rc 0, transno 4294973856 00000100:00100000:2.0:1787826339.800085:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346274944:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1787826340.013943:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826340.063735:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063784:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063822:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063845:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063869:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063886:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063907:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063924:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063950:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.063979:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332639488 00000100:00100000:0.0:1787826340.063988:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332639488 00000100:00100000:2.0:1787826340.064070:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332639232 00000100:00100000:2.0:1787826340.064075:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332639232 00000100:00100000:2.0:1787826340.064154:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2232 00000100:00100000:2.0:1787826340.064165:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79747d9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332639232:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826340.064236:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79747d9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332639232:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 73us (522us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.064244:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2232 00000100:00100000:3.0:1787826340.064384:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332640000 00000100:00100000:3.0:1787826340.064389:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332640000 00000100:00100000:3.0:1787826340.064429:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332639360 00000100:00100000:3.0:1787826340.064432:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332639360 00000100:00100000:2.0:1787826340.064478:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332639872 00000100:00100000:2.0:1787826340.064482:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332639872 00000100:00100000:2.0:1787826340.064510:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14334 00000100:00100000:2.0:1787826340.064517:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332640000:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826340.064550:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332640000:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 33us (704us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.064558:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14334 00000100:00100000:2.0:1787826340.064572:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14335 00000100:00100000:2.0:1787826340.064579:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332639872:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826340.064607:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332639872:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (682us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.064614:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14335 00000100:00100000:0.0:1787826340.064706:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14336 00000100:00100000:0.0:1787826340.064715:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332639488:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826340.064743:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332639488:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (959us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.064750:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14336 00000100:00100000:0.0:1787826340.064776:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332639744 00000100:00100000:0.0:1787826340.064780:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332639744 00000100:00100000:3.0:1787826340.066061:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14337 00000100:00100000:3.0:1787826340.066076:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332639360:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826340.066389:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332639360:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 316us (2500us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.066397:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14337 00000100:00100000:3.0:1787826340.066425:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332640256 00000100:00100000:3.0:1787826340.066432:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332640256 00000100:00100000:3.0:1787826340.066475:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332639616 00000100:00100000:3.0:1787826340.066479:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332639616 00000100:00100000:3.0:1787826340.066496:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332640128 00000100:00100000:3.0:1787826340.066498:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332640128 00000100:00100000:3.0:1787826340.066516:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8847 00000100:00100000:3.0:1787826340.066522:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332640256:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826340.066548:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332640256:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (2680us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.066554:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8847 00000100:00100000:3.0:1787826340.066566:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8848 00000100:00100000:3.0:1787826340.066572:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332639616:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826340.066596:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332639616:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (2689us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.066602:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8848 00000100:00100000:3.0:1787826340.066613:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8849 00000100:00100000:3.0:1787826340.066619:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332640128:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826340.066640:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332640128:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (2690us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.066646:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8849 00000100:00100000:0.0:1787826340.066840:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8850 00000100:00100000:0.0:1787826340.066848:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332639744:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826340.066871:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332639744:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (3050us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.066877:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8850 00000100:00080000:2.0:1787826340.069519:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826340.069542:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826340.069547:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826340.463983:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.464098:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332640384 00000100:00100000:0.0:1787826340.464132:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332640384 00000100:00100000:0.0:1787826340.464204:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68553 00000100:00100000:0.0:1787826340.464215:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e762a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225813:x1874663332640384:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826340.464236:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.464259:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b7dd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801815 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.464330:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b7e4 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.464343:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b7e4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.464353:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b7e4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826340.464465:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e54:0x0]: returning size 4096 00000004:00200000:0.0:1787826340.464471:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787826340.464491:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b7e4 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.464519:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4b7dd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801815 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.464541:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b7e4 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801815 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.464549:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:0.0:1787826340.464553:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826340.464556:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826340.464620:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e762a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225813:x1874663332640384:12345-192.168.203.56@tcp:101:rm.0 Request processed in 407us (646us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.464627:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68553 00000100:00100000:2.0:1787826340.471001:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.474991:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332640512 00000100:00100000:3.0:1787826340.475015:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332640512 00000100:00100000:3.0:1787826340.475090:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68554 00000100:00100000:3.0:1787826340.475101:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225813:x1874663332640512:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826340.475140:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826340.475165:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b7eb lrc: 2/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080181c expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.475226:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.475239:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.475250:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826340.475298:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x198:0x0]: returning size 4096 00000004:00200000:3.0:1787826340.475303:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826340.475337:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b7eb lrc: 0/0,0 mode: --/CR res: [0x240000400:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.475355:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.475363:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826340.475366:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826340.475369:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826340.475435:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225813:x1874663332640512:12345-192.168.203.56@tcp:101:rm.0 Request processed in 336us (4442us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.475456:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68554 00000100:00100000:0.0:1787826340.480695:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826340.482554:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332640640 00000100:00100000:1.0:1787826340.482568:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332640640 00000100:00100000:1.0:1787826340.482639:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68555 00000100:00100000:1.0:1787826340.482650:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225813:x1874663332640640:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826340.482671:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826340.482697:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b7f9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801823 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.482770:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.482784:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.482794:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826340.482838:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x196:0x0]: returning size 4096 00000004:00200000:1.0:1787826340.482843:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826340.482880:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b7f9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.482908:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.482917:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:1.0:1787826340.482920:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826340.482924:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826340.483017:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225813:x1874663332640640:12345-192.168.203.56@tcp:101:rm.0 Request processed in 368us (2332us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.483023:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68555 00000100:00100000:1.0:1787826340.489939:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.490019:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332640896 00000100:00100000:3.0:1787826340.490029:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332640896 00000100:00100000:3.0:1787826340.490098:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68556 00000100:00100000:3.0:1787826340.490129:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225813:x1874663332640896:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826340.490151:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826340.490177:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b807 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080182a expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826340.490340:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e54:0x0]: returning size 4096 00010000:00010000:3.0:1787826340.490429:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b807 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080182a expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.490454:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826340.490457:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826340.490462:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b807 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080182a expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.490488:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826340.490545:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225813:x1874663332640896:12345-192.168.203.56@tcp:101:rm.0 Request processed in 439us (617us total) trans 4294985440 rc 301/301 00000100:00100000:3.0:1787826340.490552:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68556 00000100:00100000:3.0:1787826340.495991:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.496128:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332641024 00000100:00100000:0.0:1787826340.496138:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332641024 00000100:00100000:0.0:1787826340.496198:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9369 00000100:00100000:0.0:1787826340.496208:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225813:x1874663332641024:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826340.500265:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225813:x1874663332641024:12345-192.168.203.56@tcp:37:rm.0 Request processed in 4048us (4275us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.500279:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9369 00000100:00100000:3.0:1787826340.506720:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.506765:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332641152 00000100:00100000:3.0:1787826340.506774:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332641152 00000100:00100000:3.0:1787826340.506824:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9370 00000100:00100000:3.0:1787826340.506833:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225813:x1874663332641152:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826340.513558:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225813:x1874663332641152:12345-192.168.203.56@tcp:37:rm.0 Request processed in 6606us (6727us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.513573:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9370 00000100:00100000:3.0:1787826340.517987:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.518125:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332641408 00000100:00100000:2.0:1787826340.518135:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332641408 00000100:00100000:2.0:1787826340.518189:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9371 00000100:00100000:2.0:1787826340.518207:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225813:x1874663332641408:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:2.0:1787826340.518531:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1215, 344 off1107464 00000040:00080000:2.0:1787826340.518586:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1534, 424 off1551464 00000004:00080000:2.0:1787826340.518591:0:6530:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7851d813c0 with version 6860 00000004:00080000:3.0:1787826340.520062:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6860 rpc_version 6860 00000020:00100000:3.0:1787826340.520102:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826340.520143:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c94000 x1874663346275584/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826340.520171:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346275584:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826340.520264:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.520301:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.524403:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346275584 00000100:00100000:2.0:1787826340.524417:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346275584 00000100:00100000:2.0:1787826340.524499:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23903 00000100:00100000:2.0:1787826340.524510:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346275584:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826340.524778:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346275584:12345-0@lo:1000:osp_up0-1.0 Request processed in 270us (4515us total) trans 8589957439 rc 0/0 00000100:00100000:2.0:1787826340.524786:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23903 00000100:00100000:2.0:1787826340.524837:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957435 gen 2 00000004:00080000:2.0:1787826340.524844:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957432 committed transno 8589957435 00000004:00080000:2.0:1787826340.524865:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957433 committed transno 8589957435 00000004:00080000:2.0:1787826340.524878:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957434 committed transno 8589957435 00000004:00080000:2.0:1787826340.524886:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957435 committed transno 8589957435 00000100:00100000:2.0:1787826340.524900:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f7842c94000 x1874663346275584/t8589957439(8589957439) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826356 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826340.526504:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346275584:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826340.526657:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225813:x1874663332641408:12345-192.168.203.56@tcp:35:rm.0 Request processed in 8457us (8678us total) trans 4294985441 rc 0/0 00000100:00100000:2.0:1787826340.526665:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9371 00000100:00100000:2.0:1787826340.531296:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.531340:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332641664 00000100:00100000:2.0:1787826340.531350:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332641664 00000100:00100000:2.0:1787826340.531417:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68557 00000100:00100000:2.0:1787826340.531428:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225813:x1874663332641664:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826340.531464:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826340.531487:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b815 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801831 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.531636:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b81c lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.531649:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b81c lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.531660:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b81c lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826340.531671:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e54:0x0]: returning size 4096 00010000:00010000:2.0:1787826340.531756:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4b815 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801831 expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.531778:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b81c lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801831 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.531787:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826340.531790:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1787826340.531793:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826340.531852:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225813:x1874663332641664:12345-192.168.203.56@tcp:101:rm.0 Request processed in 427us (566us total) trans 4294985442 rc 0/0 00000100:00100000:2.0:1787826340.531862:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68557 00000100:00100000:2.0:1787826340.541516:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.541744:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332641792 00000100:00100000:0.0:1787826340.541754:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332641792 00000100:00100000:0.0:1787826340.541823:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68558 00000100:00100000:0.0:1787826340.541835:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225813:x1874663332641792:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:0.0:1787826340.542003:0:6529:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345590272 00000100:00100000:0.0:1787826340.542085:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225813:x1874663332641792:12345-192.168.203.56@tcp:41:rm.0 Request processed in 252us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.542097:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68558 00000100:00100000:3.0:1787826340.551138:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.551180:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332641920 00000100:00100000:3.0:1787826340.551189:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332641920 00000100:00100000:3.0:1787826340.551245:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68559 00000100:00100000:3.0:1787826340.552892:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225813:x1874663332641920:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826340.553017:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826340.553040:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b82a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801838 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553124:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553138:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553150:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553165:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553171:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553178:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553284:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b83f lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553293:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b83f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553300:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b83f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826340.553321:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b83f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553334:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553340:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553346:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553353:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553359:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826340.553363:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553369:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553379:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:3.0:1787826340.553382:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826340.553386:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553391:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b831 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553402:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553407:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553412:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553417:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826340.553419:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553425:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553431:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f79785cc400) 00010000:00010000:3.0:1787826340.553433:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826340.553435:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553440:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4b838 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553478:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4b82a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801838 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553491:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b83f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801838 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.553503:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f79785ccb00) 00010000:00010000:3.0:1787826340.553506:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826340.553508:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826340.554726:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225813:x1874663332641920:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1828us (3599us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.554738:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68559 00000100:00100000:0.0:1787826340.572122:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.572165:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332642048 00000100:00100000:2.0:1787826340.572175:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332642048 00000100:00100000:2.0:1787826340.572558:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68560 00000100:00100000:2.0:1787826340.572570:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:225813:x1874663332642048:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826340.572593:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826340.572617:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b846 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080183f expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.572683:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b84d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.572696:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b84d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.572708:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b84d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826340.572760:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e55:0x0]: returning size 6 00000004:00200000:2.0:1787826340.572764:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:2.0:1787826340.572803:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4b846 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080183f expref: 14 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.572820:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b84d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080183f expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.572829:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f7967ac8300) 00010000:00010000:2.0:1787826340.572832:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826340.572835:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826340.572898:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225813:x1874663332642048:12345-192.168.203.56@tcp:101:rm.0 Request processed in 331us (800us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.572906:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68560 00000100:00100000:0.0:1787826340.579698:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.579851:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.579883:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332642176 00000100:00100000:2.0:1787826340.579894:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332642176 00000100:00100000:3.0:1787826340.579900:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332642432 00000100:00100000:3.0:1787826340.579904:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332642432 00000100:00100000:2.0:1787826340.579958:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11153 00000100:00100000:3.0:1787826340.579958:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68561 00000100:00100000:2.0:1787826340.579970:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b470050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332642176:12345-192.168.203.56@tcp:103:rm.0 00000100:00100000:3.0:1787826340.579978:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225813:x1874663332642432:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:2.0:1787826340.579986:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826340.580017:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4b83f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801838 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.580028:0:6527:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826340.580036:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f79785ccb00) 00010000:00010000:3.0:1787826340.580042:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4b84d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e55:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080183f expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.580046:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826340.580054:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f7967ac8300) 00010000:00010000:3.0:1787826340.580061:0:6527:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826340.580126:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b470050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332642176:12345-192.168.203.56@tcp:103:rm.0 Request processed in 140us (418us total) trans 0 rc 0/0 00010000:00010000:3.0:1787826340.580135:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.580138:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11153 00010000:00010000:3.0:1787826340.580148:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.580157:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.580179:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346275712:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787826340.580219:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.580253:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.580402:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346275712 00000100:00100000:2.0:1787826340.580407:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346275712 00000100:00100000:1.0:1787826340.582340:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23904 00000100:00100000:1.0:1787826340.582361:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346275712:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787826340.582385:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826340.582407:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b85b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b854 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.582436:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.582460:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b85b lrc: 4/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b854 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.582494:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.582509:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 9 pid: 17813 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826340.582522:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346275840:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826340.582616:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.585946:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.586708:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332642560 00000100:00100000:2.0:1787826340.586718:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332642560 00010000:00010000:2.0:1787826340.586836:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 10 pid: 17813 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.586911:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22006 00000100:00100000:2.0:1787826340.586922:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332642560:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826340.586937:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826340.586944:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 10 pid: 17813 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.586959:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 10 pid: 17813 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.586984:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:2.0:1787826340.586990:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b85b lrc: 3/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b854 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.587001:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b85b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4b854 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.587009:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b85b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4b854 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.587047:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b85b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b854 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.587062:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346275968:0@lo:105: 00000100:00100000:2.0:1787826340.587237:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.587284:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.587340:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346275968 00010000:00010000:1.0:1787826340.587342:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4b800 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x196:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801823 expref: 10 pid: 17813 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.587346:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346275968 00000100:00100000:1.0:1787826340.587370:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346275840:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826340.587392:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7893 00010000:00010000:1.0:1787826340.587398:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b85b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b854 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.587403:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346275968:12345-0@lo:105: 00010000:00010000:1.0:1787826340.587415:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:1.0:1787826340.587420:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.587422:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.587423:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00010000:00010000:0.0:1787826340.587435:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.587454:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.587462:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.587470:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.587478:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826340.587489:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346275712:12345-0@lo:101:mdt00_000.0 Request processed in 5135us (7268us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.587498:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23904 00010000:00010000:0.0:1787826340.587515:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68e00) 00000100:00100000:1.0:1787826340.587553:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346275712:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787826340.587560:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346275968:12345-0@lo:105: Request processed in 158us (320us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.587569:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7893 00010000:00010000:1.0:1787826340.587575:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826340.587598:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x196:0x0] 00010000:00010000:1.0:1787826340.587641:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.587649:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.587657:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.587754:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346275968:0@lo:105: 00010000:00010000:2.0:1787826340.587765:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826340.587809:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332642560:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 890us (1872us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.587817:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22006 00000100:00100000:1.0:1787826340.589032:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276096:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787826340.589086:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.589133:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aa300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.589283:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346276096 00000100:00100000:2.0:1787826340.589290:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346276096 00000100:00100000:2.0:1787826340.589369:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23905 00000100:00100000:2.0:1787826340.589378:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346276096:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787826340.589396:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826340.589417:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4b869 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b862 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.589441:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4b869 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b862 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.589464:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4b869 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b862 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.589472:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f7967ac8f00) 00010000:00010000:2.0:1787826340.589475:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826340.589478:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826340.589534:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346276096:12345-0@lo:101:mdt00_000.0 Request processed in 156us (447us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.589541:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23905 00000100:00100000:1.0:1787826340.589607:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276096:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787826340.589632:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b869 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.589643:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b869 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826340.589670:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x196:0x0] 00000100:00100000:1.0:1787826340.589708:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276224:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826340.589734:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.589766:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500aae40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.589832:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346276224 00000100:00100000:3.0:1787826340.589838:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346276224 00000100:00100000:3.0:1787826340.589872:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23906 00000100:00100000:3.0:1787826340.589880:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346276224:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826340.589972:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346276224:12345-0@lo:1000:mdt00_000.0 Request processed in 84us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.589978:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23906 00000100:00100000:0.0:1787826340.590047:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276224:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826340.590100:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465ca300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276352:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826340.590156:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.590169:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.590276:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346276352 00000100:00100000:3.0:1787826340.590280:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346276352 00000100:00100000:3.0:1787826340.590306:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23907 00000100:00100000:3.0:1787826340.590313:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346276352:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826340.590385:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346276352:12345-0@lo:1000:mdt00_000.0 Request processed in 72us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.590392:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23907 00000100:00100000:0.0:1787826340.590467:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465ca300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276352:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826340.590666:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465ca300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276480:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826340.590694:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.590718:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.590823:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346276480 00000100:00100000:3.0:1787826340.590828:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346276480 00000100:00100000:3.0:1787826340.590861:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23908 00000100:00100000:3.0:1787826340.590868:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346276480:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826340.590942:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346276480:12345-0@lo:1000:mdt00_000.0 Request processed in 75us (246us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826340.590949:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23908 00000100:00100000:3.0:1787826340.590976:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465ca300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276480:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1787826340.591031:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591042:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591051:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.591064:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276608:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787826340.591085:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.591097:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb79c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.591121:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346276608 00000100:00100000:3.0:1787826340.591124:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346276608 00000100:00100000:3.0:1787826340.591197:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23909 00000100:00100000:3.0:1787826340.591205:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346276608:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787826340.591212:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826340.591225:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b877 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b870 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591244:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b877 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b870 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591254:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b877 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b870 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591262:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f79785cca00) 00010000:00010000:3.0:1787826340.591265:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826340.591268:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826340.591296:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6527:x1874663346276608:12345-0@lo:101:mdt00_000.0 Request processed in 92us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.591302:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23909 00000100:00100000:3.0:1787826340.591327:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346276608:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787826340.591339:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591350:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591641:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591653:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.591662:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787826340.592368:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].224, 64 off22528 00000004:00080000:3.0:1787826340.592381:0:6527:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8152804274965ns, next at 8153804272285ns, rc = 0 00000040:00080000:3.0:1787826340.592412:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1216, 384 off1107848 00000040:00080000:3.0:1787826340.592469:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1535, 464 off1551928 00000004:00080000:3.0:1787826340.592473:0:6527:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784afb79c0 with version 6861 00000004:00080000:2.0:1787826340.592605:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6861 rpc_version 6861 00000020:00100000:2.0:1787826340.592629:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826340.592635:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9eaa00 x1874663346276864/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826340.592649:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346276864:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826340.592904:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.592976:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.593055:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346276864 00000100:00100000:3.0:1787826340.593061:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346276864 00000100:00100000:3.0:1787826340.593129:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23910 00000100:00100000:3.0:1787826340.593144:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346276864:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826340.593390:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346276864:12345-0@lo:1000:osp_up0-1.0 Request processed in 254us (486us total) trans 8589957440 rc 0/0 00000100:00100000:3.0:1787826340.593395:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23910 00010000:00010000:1.0:1787826340.593537:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.593553:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.593565:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.593573:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.593580:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.593586:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.593614:0:6527:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826340.593622:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346276864:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826340.593755:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826340.593762:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.593770:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826340.593774:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.593782:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.593795:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:2.0:1787826340.593798:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826340.593803:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.593809:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4b885 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e55:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.593825:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826340.593826:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826340.593828:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826340.593991:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.593998:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594004:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594010:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826340.594014:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594020:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594030:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:1.0:1787826340.594032:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826340.594036:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594041:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4b87e lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594059:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594068:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594075:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594124:0:6527:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826340.594137:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x196:0x0] 00010000:00010000:1.0:1787826340.594147:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594155:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594161:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594319:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826340.594325:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594333:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826340.594338:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826340.594345:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594375:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594381:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826340.594410:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594417:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4b870 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b877 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594431:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826340.594433:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826340.594435:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826340.594479:0:6527:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826340.594484:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x196:0x0] 00010000:00010000:1.0:1787826340.594489:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b869 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.594497:0:6527:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b869 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826340.594570:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225813:x1874663332642432:12345-192.168.203.56@tcp:36:rm.0 Request processed in 14599us (14715us total) trans 4294985443 rc 0/0 00000100:00100000:1.0:1787826340.594577:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68561 00000100:00100000:0.0:1787826340.594653:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:2.0:1787826340.594822:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826340.594828:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594836:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826340.594841:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826340.594850:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594872:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594878:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826340.594903:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.594911:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68e00/0xe21309aa81f4b854 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b85b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.594986:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346276992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826340.595034:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.595837:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346277120:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826340.595882:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826340.596037:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826340.596040:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826340.596042:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346276992 00010000:00010000:2.0:1787826340.596043:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826340.596051:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346276992 00000100:00100000:3.0:1787826340.596122:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346277120 00000100:00100000:3.0:1787826340.596125:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346277120 00000100:00100000:0.0:1787826340.596145:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11154 00000100:00100000:3.0:1787826340.596153:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11155 00000100:00100000:0.0:1787826340.596159:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346276992:12345-0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826340.596165:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346277120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.596190:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.596211:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4b85b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x196:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b854 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.596227:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:3.0:1787826340.596235:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.596268:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346277120:12345-0@lo:103:ldlm_bl.0 Request processed in 107us (386us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.596274:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11155 00010000:00010000:0.0:1787826340.596293:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826340.596307:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4b877 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e55:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b870 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.596332:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e55:0x0].0x0 (ffff9f79785cca00) 00010000:00010000:0.0:1787826340.596340:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826340.596379:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346276992:12345-0@lo:103:ldlm_bl.0 Request processed in 225us (1344us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.596386:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11154 00000100:00100000:1.0:1787826340.596583:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346277120:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826340.596651:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346276992:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826340.599849:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.599900:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.599914:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332642688 00000100:00100000:3.0:1787826340.599926:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332642688 00000100:00100000:3.0:1787826340.600004:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11156 00000100:00100000:3.0:1787826340.600016:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332642688:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826340.600031:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.600058:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b6b0 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017ba expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.600076:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:3.0:1787826340.600080:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.600380:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332642688:12345-192.168.203.56@tcp:103:rm.0 Request processed in 365us (539us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.600388:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11156 00000100:00100000:1.0:1787826340.600926:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332642944 00000100:00100000:1.0:1787826340.600934:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332642944 00000100:00100000:1.0:1787826340.601371:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9372 00000100:00100000:1.0:1787826340.601381:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:225813:x1874663332642944:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:1.0:1787826340.601612:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:225813:x1874663332642944:12345-192.168.203.56@tcp:35:rm.0 Request processed in 231us (1710us total) trans 4294985444 rc 0/0 00000100:00100000:1.0:1787826340.601631:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9372 00000100:00100000:0.0:1787826340.605204:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826340.605356:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332643200 00000100:00100000:1.0:1787826340.605366:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332643200 00000100:00100000:1.0:1787826340.605437:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68562 00000100:00100000:1.0:1787826340.605461:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:225813:x1874663332643200:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826340.605526:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826340.605556:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4b81c lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e54:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801831 expref: 14 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.605598:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b7e4 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801815 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.605614:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4b6be lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017c1 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.605626:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:1.0:1787826340.605631:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826340.605693:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.605702:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.605718:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826340.605744:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346277248:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1787826340.605979:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.606020:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b482a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.606080:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346277248 00000100:00100000:0.0:1787826340.606086:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346277248 00000100:00100000:0.0:1787826340.606319:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23911 00000100:00100000:0.0:1787826340.606329:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346277248:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826340.606343:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.606359:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b88c expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.606383:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.606392:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.606402:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b88c expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.606426:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.606437:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 7 pid: 6528 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.606465:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b618a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346277376:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826340.606547:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.606556:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 7 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.606565:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346277504:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826340.606586:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.614440:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.614499:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826340.614577:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332643328 00000100:00100000:1.0:1787826340.614587:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332643328 00000100:00100000:2.0:1787826340.614623:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332643456 00000100:00100000:2.0:1787826340.614629:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332643456 00010000:00010000:1.0:1787826340.614639:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 8 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.614652:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 8 pid: 6528 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826340.614688:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22007 00000100:00100000:1.0:1787826340.614699:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332643328:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826340.614711:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826340.614719:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 8 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.614730:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 8 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.614752:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826340.614760:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b88c expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.614771:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826340.614831:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332643328:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 133us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.614833:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22008 00000100:00100000:1.0:1787826340.614849:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22007 00000100:00100000:2.0:1787826340.614849:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332643456:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826340.614862:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826340.614869:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 8 pid: 6528 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.614882:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 8 pid: 6528 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.614896:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826340.614901:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b88c expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.615186:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b6fd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017dd expref: 8 pid: 6528 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.615206:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b618a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346277376:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826340.615221:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4b694 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017b3 expref: 7 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.615233:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346277504:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826340.615342:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4b88c expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.615352:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4b88c expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.618262:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b88c expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.618349:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826340.618352:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.618357:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826340.618460:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:17813:x1874663346277248:12345-0@lo:101:mdt00_005.0 Request processed in 12130us (12478us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.618471:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23911 00000100:00100000:1.0:1787826340.618491:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346277248:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787826340.618502:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b88c expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.618516:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.618530:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.618532:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346277632:0@lo:105: 00000100:00100000:3.0:1787826340.618580:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.618644:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346277632 00000100:00100000:1.0:1787826340.618651:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346277632 00000100:00100000:1.0:1787826340.618714:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7894 00000100:00100000:1.0:1787826340.618723:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346277632:12345-0@lo:105: 00000100:00100000:3.0:1787826340.618728:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785180 going to sleep for 11 seconds 00010000:00010000:1.0:1787826340.618735:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.618746:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.618763:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.618771:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.618780:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.618794:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.618821:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf800) 00000100:00100000:1.0:1787826340.618867:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346277632:12345-0@lo:105: Request processed in 145us (287us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.618873:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7894 00010000:00010000:2.0:1787826340.619096:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.619135:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826340.619171:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826340.619186:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346277632:0@lo:105: 00010000:00010000:3.0:1787826340.619199:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826340.619212:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.619244:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332643456:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 4400us (4743us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.619254:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22008 00010000:00010000:2.0:1787826340.619331:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.619343:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.619358:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346277760:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1787826340.619389:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.619418:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.619478:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346277760 00000100:00100000:3.0:1787826340.619484:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346277760 00000100:00100000:3.0:1787826340.619574:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23912 00000100:00100000:3.0:1787826340.619582:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:17813:x1874663346277760:12345-0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787826340.619597:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826340.619613:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b8a1 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b89a expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.625834:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b8a1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b89a expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.625854:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b8a1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b89a expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.625863:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cc600) 00010000:00010000:3.0:1787826340.625866:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826340.625871:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826340.629434:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346277760:12345-0@lo:101:mdt00_005.0 Request processed in 9842us (10033us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.629552:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346277760:0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826340.629591:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.629609:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.629631:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23912 00000004:00080000:1.0:1787826340.629636:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787826340.629700:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346277888:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826340.629858:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.629897:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784b482600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.630009:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346277888 00000100:00100000:0.0:1787826340.630017:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346277888 00000100:00100000:0.0:1787826340.630067:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23913 00000100:00100000:0.0:1787826340.630077:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346277888:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826340.630202:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346277888:12345-0@lo:1000:mdt00_005.0 Request processed in 126us (343us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.630209:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23913 00000100:00100000:2.0:1787826340.630609:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346277888:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787826340.630661:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346278016:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787826340.630690:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.630711:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.630890:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346278016 00000100:00100000:3.0:1787826340.630895:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346278016 00000100:00100000:3.0:1787826340.630934:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23914 00000100:00100000:3.0:1787826340.630942:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346278016:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787826340.631059:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346278016:12345-0@lo:1000:mdt00_005.0 Request processed in 117us (367us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826340.631066:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23914 00000100:00100000:2.0:1787826340.631088:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346278016:0@lo:1000:mdt00_005.0 00010000:00010000:2.0:1787826340.631314:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.631325:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.631336:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.631350:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346278144:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1787826340.631377:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.631443:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.631548:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346278144 00000100:00100000:0.0:1787826340.631553:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346278144 00000100:00100000:0.0:1787826340.631675:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23915 00000100:00100000:0.0:1787826340.631685:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:17813:x1874663346278144:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826340.631694:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.631708:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b8af lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b8a8 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.631731:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.631740:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b8af lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b8a8 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.631764:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.631776:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 6 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.631792:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346278272:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826340.631860:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826340.636603:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.636716:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332643584 00000100:00100000:2.0:1787826340.636725:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332643584 00010000:00010000:2.0:1787826340.636780:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 5/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 7 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.636831:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22009 00000100:00100000:2.0:1787826340.636843:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b472850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332643584:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826340.636857:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826340.636865:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 5/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 7 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.636880:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 7 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.636905:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:2.0:1787826340.636925:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b8af lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b8a8 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.636936:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b8af lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4b8a8 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.636945:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b8af lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4b8a8 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.636979:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b8af lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b8a8 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.636994:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346278400:0@lo:105: 00000100:00100000:2.0:1787826340.637043:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.637091:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.637159:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826340.637173:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346278400 00000100:00100000:3.0:1787826340.637177:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346278400 00000100:00100000:3.0:1787826340.637472:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7895 00000100:00100000:3.0:1787826340.637495:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797ccaac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346278400:12345-0@lo:105: 00010000:00010000:3.0:1787826340.637508:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.637515:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4b719 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608017e4 expref: 7 pid: 8752 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.637520:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.637537:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.637544:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346278272:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826340.637548:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.637562:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.637578:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b8af lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b8a8 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.637578:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.637601:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:0.0:1787826340.637606:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826340.637609:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00010000:00010000:3.0:1787826340.637610:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ee400) 00000100:00100000:3.0:1787826340.637667:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797ccaac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346278400:12345-0@lo:105: Request processed in 184us (623us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.637674:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7895 00000100:00100000:0.0:1787826340.637742:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:17813:x1874663346278144:12345-0@lo:101:mdt00_005.0 Request processed in 6059us (6364us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.637825:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346278400:0@lo:105: 00010000:00010000:2.0:1787826340.637837:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826340.637877:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b472850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332643584:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1037us (1285us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.637884:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22009 00000100:00100000:0.0:1787826340.638021:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23915 00000100:00100000:0.0:1787826340.638066:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346278144:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826340.638081:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.638159:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.638284:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.638293:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.638313:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,1 mode: --/EX res: [0x240000400:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.638322:0:17813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.638330:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 4/0,1 mode: --/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.638354:0:17813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.638366:0:17813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 11 pid: 6527 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.638379:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001_UUID:17813:1874663346278528:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826340.638437:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826340.641679:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.641747:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332643712 00000100:00100000:3.0:1787826340.641755:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332643712 00010000:00010000:3.0:1787826340.641796:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 12 pid: 6527 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.641836:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22010 00000100:00100000:3.0:1787826340.641845:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332643712:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.641858:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.641865:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 12 pid: 6527 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.641878:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 12 pid: 6527 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.641899:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826340.641905:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,1 mode: --/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.641914:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.641921:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.641940:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.642044:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332643712:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 199us (374us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.642052:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22010 00010000:00010000:0.0:1787826340.642287:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4b7f2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080181c expref: 11 pid: 6527 timeout: 8252 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.642304:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001_UUID:17813:1874663346278528:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826340.642328:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.642336:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.642342:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.642452:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.642460:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.642471:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.642538:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346278656:0@lo:101:mdt00_005.0 00000100:00100000:0.0:1787826340.642581:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.642610:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.642763:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346278656 00000100:00100000:2.0:1787826340.642771:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346278656 00000100:00100000:2.0:1787826340.642818:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23916 00000100:00100000:2.0:1787826340.642828:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:17813:x1874663346278656:12345-0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826340.642842:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826340.642860:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b8cb lrc: 2/0,0 mode: --/EX res: [0x200000401:0x196:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b8c4 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.642883:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b8cb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b8c4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.642895:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b8cb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b8c4 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.642903:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac8a00) 00010000:00010000:2.0:1787826340.642906:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826340.642909:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826340.642961:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:17813:x1874663346278656:12345-0@lo:101:mdt00_005.0 Request processed in 135us (378us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.642968:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23916 00000100:00100000:0.0:1787826340.644679:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346278656:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826340.644707:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.644720:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.645462:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346278784:0@lo:403:mdt00_005.0 00000100:00100000:0.0:1787826340.645533:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.645562:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.645839:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346278784 00000100:00100000:3.0:1787826340.645847:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346278784 00000100:00100000:3.0:1787826340.645901:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23917 00000100:00100000:3.0:1787826340.645910:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:17813:x1874663346278784:12345-0@lo:403:mdt00_005.0 00000100:00100000:2.0:1787826340.648271:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:17813:x1874663346278784:12345-0@lo:403:mdt00_005.0 Request processed in 2352us (2728us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.648283:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23917 00000100:00100000:0.0:1787826340.648361:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346278784:0@lo:403:mdt00_005.0 00000040:00080000:1.0:1787826340.650913:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1217, 952 off1108800 00000040:00080000:1.0:1787826340.651142:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1536, 1032 off1552960 00000004:00080000:1.0:1787826340.651149:0:17813:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784b4823c0 with version 6862 00000004:00080000:0.0:1787826340.651318:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6862 rpc_version 6862 00000020:00100000:0.0:1787826340.651354:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826340.651365:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b61ad80 x1874663346279040/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826340.651397:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346279040:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826340.651479:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.651517:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.651585:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346279040 00000100:00100000:3.0:1787826340.651594:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346279040 00000100:00100000:3.0:1787826340.651658:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23918 00000100:00100000:3.0:1787826340.651668:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346279040:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826340.652090:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346279040:12345-0@lo:1000:osp_up0-1.0 Request processed in 423us (609us total) trans 8589957441 rc 0/0 00000100:00100000:3.0:1787826340.652099:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23918 00000100:00100000:1.0:1787826340.652336:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346279040:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787826340.652897:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e54:0x0]: returning size 4096 00010000:00010000:0.0:1787826340.652921:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.652931:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.652939:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.652945:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826340.652951:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.652959:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.652974:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x198:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:0.0:1787826340.652977:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826340.652981:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 2/0,0 mode: --/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.652988:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4b8bd lrc: 0/0,0 mode: --/EX res: [0x240000400:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653010:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653024:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653032:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653059:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826340.653070:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x196:0x0] 00010000:00010000:0.0:1787826340.653077:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653083:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653089:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653094:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826340.653097:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653137:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653146:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:0.0:1787826340.653149:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.653153:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653159:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4b8b6 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653169:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653176:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653182:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653199:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787826340.653210:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826340.653215:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653221:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653228:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653248:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787826340.653252:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826340.653256:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.653264:0:17813:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.653344:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225813:x1874663332643200:12345-192.168.203.56@tcp:36:rm.0 Request processed in 47895us (48147us total) trans 4294985445 rc 0/0 00000100:00100000:0.0:1787826340.653352:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68562 00010000:00010000:3.0:1787826340.653423:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826340.653430:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653437:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826340.653442:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826340.653459:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653488:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653496:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826340.653526:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653534:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43000/0xe21309aa81f4b8c4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b8cb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653548:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826340.653551:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653557:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826340.653561:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826340.653567:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653582:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653588:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826340.653602:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653609:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4b8a8 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b8af expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653622:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826340.653625:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653632:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826340.653635:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826340.653641:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653657:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653662:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826340.653668:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653674:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4b88c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b893 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.653684:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826340.653686:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826340.653687:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826340.653702:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346279424:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826340.653749:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.653782:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826340.653787:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346279296:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826340.653805:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.653827:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826340.653834:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346279168:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826340.653857:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.653889:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346279424 00000100:00100000:3.0:1787826340.653895:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346279424 00000100:00100000:3.0:1787826340.653932:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346279296 00000100:00100000:3.0:1787826340.653935:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346279296 00000100:00100000:3.0:1787826340.653953:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346279168 00000100:00100000:3.0:1787826340.653956:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346279168 00000100:00100000:3.0:1787826340.653974:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11157 00000100:00100000:3.0:1787826340.653982:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+17:3653:x1874663346279424:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.653993:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.654008:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b893 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b88c expref: 17 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.654019:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826340.654023:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.654051:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+16:3653:x1874663346279424:12345-0@lo:103:ldlm_bl.0 Request processed in 70us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.654057:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11157 00000100:00100000:3.0:1787826340.654069:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11158 00000100:00100000:3.0:1787826340.654075:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+15:3653:x1874663346279296:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.654082:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.654092:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4b8af lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e54:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b8a8 expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.654125:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f79785cc300) 00010000:00010000:3.0:1787826340.654134:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.654156:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+14:3653:x1874663346279296:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.654162:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11158 00000100:00100000:3.0:1787826340.654176:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11159 00000100:00100000:3.0:1787826340.654182:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346279168:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.654189:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.654199:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4b8cb lrc: 0/0,0 mode: --/EX res: [0x200000401:0x196:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b8c4 expref: 13 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.654210:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x0 (ffff9f7967ac8a00) 00010000:00010000:3.0:1787826340.654216:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.654241:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346279168:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.654247:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11159 00000100:00100000:3.0:1787826340.654288:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346279296:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826340.654298:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346279424:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826340.654320:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346279168:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826340.654555:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826340.657127:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826340.741641:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.741784:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332643840 00000100:00100000:3.0:1787826340.741795:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332643840 00000100:00100000:3.0:1787826340.741857:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68563 00000100:00100000:3.0:1787826340.741868:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225821:x1874663332643840:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:3.0:1787826340.741899:0:8752:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345590272 00000100:00100000:3.0:1787826340.741974:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225821:x1874663332643840:12345-192.168.203.56@tcp:41:stat.0 Request processed in 109us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.741980:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68563 00000100:00100000:1.0:1787826340.807317:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.807570:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332643968 00000100:00100000:2.0:1787826340.807582:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332643968 00000100:00100000:2.0:1787826340.807652:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68564 00000100:00100000:2.0:1787826340.807663:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225825:x1874663332643968:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826340.807685:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826340.807709:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4b8d2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801846 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.808317:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.808332:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.808346:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826340.808398:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826340.808403:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826340.808424:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.808463:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4b8d2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.808484:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.808493:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826340.808497:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826340.808501:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826340.808629:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225825:x1874663332643968:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 967us (1321us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.808638:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68564 00000100:00100000:2.0:1787826340.817456:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.817538:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332644096 00000100:00100000:3.0:1787826340.817548:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332644096 00000100:00100000:3.0:1787826340.817614:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68565 00000100:00100000:3.0:1787826340.817626:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225825:x1874663332644096:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826340.817647:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826340.817669:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b8e0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080184d expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.817744:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.817758:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.817770:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.817782:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.817793:0:24703:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b8a1 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b89a expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.817802:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.817840:0:24703:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b8a1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b89a expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.817857:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346279552:0@lo:104: 00000100:00100000:3.0:1787826340.817906:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.817950:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.818202:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346279552 00000100:00100000:2.0:1787826340.818207:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346279552 00000100:00100000:2.0:1787826340.818248:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7896 00000100:00100000:2.0:1787826340.818256:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346279552:12345-0@lo:104: 00010000:00010000:2.0:1787826340.818268:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.818410:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826340.818417:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346279552:12345-0@lo:104: Request processed in 162us (510us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.818423:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7896 00010000:00010000:1.0:1787826340.818473:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826340.818480:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.818488:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826340.818494:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826340.818503:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.818529:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.818535:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826340.818562:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.818570:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4b89a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b8a1 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.818586:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826340.818588:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826340.818589:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826340.818606:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826340.818616:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346279680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826340.818650:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.819294:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346279680 00000100:00100000:3.0:1787826340.819300:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346279680 00010000:00010000:3.0:1787826340.819324:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b8a1 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b89a expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.819357:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22011 00000100:00100000:3.0:1787826340.819364:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346279680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.819376:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787826340.822246:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346279552:0@lo:104: 00010000:00010000:0.0:1787826340.822284:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826340.825764:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957441 is committed 00010000:00010000:1.0:1787826340.832689:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b8a1 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b89a expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.832772:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cc600) 00010000:00010000:1.0:1787826340.832779:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.832789:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.832797:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.832875:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826340.834975:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346279680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826340.835374:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346279680:12345-0@lo:103:ldlm_bl.0 Request processed in 13601us (14314us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.835386:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22011 00010000:00010000:0.0:1787826340.836798:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.836813:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.836887:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.836895:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.836901:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.836908:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.836913:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826340.836917:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.836929:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.836947:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826340.836950:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.836955:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837568:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b8e7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837593:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837599:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837605:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837610:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826340.837613:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837619:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837628:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cc600) 00010000:00010000:0.0:1787826340.837631:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.837634:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837644:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b8ee lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837693:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b8e0 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080184d expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.837705:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826340.837707:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.838031:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4b8e0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080184d expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.838304:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:0.0:1787826340.838422:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225825:x1874663332644096:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 20795us (20982us total) trans 0 rc 301/301 00000100:00100000:0.0:1787826340.838439:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68565 00000100:00100000:3.0:1787826340.849574:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826340.849683:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346279936:0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.849765:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.849807:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826340.849814:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346280064:0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.849838:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.849871:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346279936 00000100:00100000:1.0:1787826340.849878:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346279936 00000100:00100000:1.0:1787826340.849930:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346280064 00000100:00100000:1.0:1787826340.849933:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346280064 00000100:00100000:1.0:1787826340.849971:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14338 00000100:00100000:1.0:1787826340.849981:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346279936:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.850029:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346279936:12345-0@lo:400:kworker.0 Request processed in 49us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.850037:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14338 00000100:00100000:1.0:1787826340.850054:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14339 00000100:00100000:1.0:1787826340.850061:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346280064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.850086:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346280064:12345-0@lo:400:kworker.0 Request processed in 26us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.850091:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14339 00000100:00100000:1.0:1787826340.850155:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346280064:0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.850167:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346279936:0@lo:400:kworker.0 00000100:00080000:2.0:1787826340.851497:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826340.851510:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826340.851528:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346280192:0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.851583:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.851707:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346279808:0@lo:400:kworker.0 00000100:00100000:3.0:1787826340.851755:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.852258:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346280192 00000100:00100000:1.0:1787826340.852266:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346280192 00000100:00100000:1.0:1787826340.852322:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14340 00000100:00100000:1.0:1787826340.852332:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346280192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.852393:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346280192:12345-0@lo:400:kworker.0 Request processed in 62us (808us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.852394:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346279808 00000100:00100000:2.0:1787826340.852400:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346279808 00000100:00100000:1.0:1787826340.852404:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14340 00000100:00100000:2.0:1787826340.852455:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2233 00000100:00100000:2.0:1787826340.852464:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79747d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346279808:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.852515:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79747d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346279808:12345-0@lo:400:kworker.0 Request processed in 53us (759us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.852523:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2233 00000100:00080000:2.0:1787826340.852550:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826340.852585:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346279808:0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.852793:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346280192:0@lo:400:kworker.0 00000100:00100000:3.0:1787826340.852907:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826340.853219:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332644224 00000100:00100000:1.0:1787826340.853224:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332644224 00000100:00100000:1.0:1787826340.853265:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68566 00000100:00100000:1.0:1787826340.853273:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225825:x1874663332644224:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826340.853288:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826340.853413:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b8f5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801854 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.853492:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.853506:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.853517:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826340.853566:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826340.853570:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826340.853589:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.853616:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4b8f5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.853633:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.853642:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826340.853645:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826340.853648:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826340.853701:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225825:x1874663332644224:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 429us (797us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.853709:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68566 00000100:00100000:1.0:1787826340.855427:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346280320:0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.855509:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.855587:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346280320 00000100:00100000:3.0:1787826340.855594:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346280320 00000100:00100000:3.0:1787826340.855656:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8851 00000100:00100000:3.0:1787826340.855666:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3655:x1874663346280320:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1787826340.857741:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826340.858036:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3655:x1874663346280320:12345-0@lo:400:kworker.0 Request processed in 2365us (2520us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.858050:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8851 00000100:00100000:3.0:1787826340.858180:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346280448:0@lo:400:kworker.0 00000100:00100000:3.0:1787826340.858244:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.858276:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826340.858284:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346280576:0@lo:400:kworker.0 00000100:00100000:3.0:1787826340.858310:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.858331:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346280960:0@lo:400:kworker.0 00000100:00100000:0.0:1787826340.858345:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346281088:0@lo:400:kworker.0 00000100:00100000:3.0:1787826340.858378:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.858394:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346280448 00000100:00100000:0.0:1787826340.858401:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346280448 00000100:00100000:3.0:1787826340.858427:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346280960 00000100:00100000:3.0:1787826340.858432:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346280960 00000100:00100000:0.0:1787826340.858470:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346280576 00000100:00100000:0.0:1787826340.858474:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346280576 00000100:00100000:3.0:1787826340.858478:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8852 00000100:00100000:3.0:1787826340.858488:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346280960:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826340.858502:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14341 00000100:00100000:0.0:1787826340.858512:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346280448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826340.858539:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346280960:12345-0@lo:400:kworker.0 Request processed in 53us (161us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.858547:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8852 00000100:00100000:0.0:1787826340.858555:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346280448:12345-0@lo:400:kworker.0 Request processed in 44us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.858563:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14341 00000100:00100000:0.0:1787826340.858579:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14342 00000100:00100000:3.0:1787826340.858582:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346280704:0@lo:400:kworker.0 00000100:00100000:0.0:1787826340.858588:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346280576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826340.858617:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346280576:12345-0@lo:400:kworker.0 Request processed in 32us (307us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.858623:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14342 00000100:00080000:0.0:1787826340.858637:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787826340.858639:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826340.858653:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.858685:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346281088 00000100:00100000:0.0:1787826340.858688:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346281088 00000100:00100000:0.0:1787826340.858831:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8853 00000100:00100000:0.0:1787826340.858840:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346281088:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826340.858873:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346281088:12345-0@lo:400:kworker.0 Request processed in 35us (219us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.858878:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8853 00000100:00080000:0.0:1787826340.858891:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826340.858926:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976163 gen 2 00000004:00080000:0.0:1787826340.858931:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784d8be300, transno 4294976163 00000100:00100000:0.0:1787826340.858943:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346281088:0@lo:400:kworker.0 00000100:00100000:0.0:1787826340.859134:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346280960:0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.859301:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.859326:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346280576:0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.859331:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332644480 00000100:00100000:2.0:1787826340.859336:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332644480 00000100:00100000:0.0:1787826340.859340:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346280448:0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.859373:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68567 00000100:00100000:2.0:1787826340.859383:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225825:x1874663332644480:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:2.0:1787826340.859510:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346281344:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826340.859537:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346280320:0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.859551:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826340.859558:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346280832:0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.859584:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826340.859593:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.859818:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.859847:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346281216:0@lo:400:kworker.0 00000100:00100000:3.0:1787826340.859982:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.860011:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346280704 00000100:00100000:3.0:1787826340.860015:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346280704 00000100:00100000:1.0:1787826340.860070:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346281216 00000100:00100000:1.0:1787826340.860075:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346281216 00000100:00100000:1.0:1787826340.860128:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14343 00000100:00100000:1.0:1787826340.860135:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346281216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.860191:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346281216:12345-0@lo:400:kworker.0 Request processed in 55us (207us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.860197:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14343 00000100:00100000:3.0:1787826340.860218:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8854 00000100:00100000:3.0:1787826340.860228:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346280704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826340.860241:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346281216:0@lo:400:kworker.0 00000100:00100000:3.0:1787826340.860276:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346280704:12345-0@lo:400:kworker.0 Request processed in 50us (458us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.860283:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8854 00000100:00100000:1.0:1787826340.860284:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976162 gen 2 00000004:00080000:1.0:1787826340.860290:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784d816a00, transno 4294976162 00000100:00100000:1.0:1787826340.860302:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346280704:0@lo:400:kworker.0 00000100:00080000:1.0:1787826340.861169:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826340.861543:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346281344 00000100:00100000:2.0:1787826340.861550:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346281344 00000100:00100000:2.0:1787826340.861580:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346280832 00000100:00100000:2.0:1787826340.861583:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346280832 00000100:00100000:2.0:1787826340.861639:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23919 00000100:00100000:2.0:1787826340.861649:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346281344:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826340.865836:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346281344:12345-0@lo:1000:mdt00_002.0 Request processed in 4178us (6275us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.865851:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23919 00000100:00100000:2.0:1787826340.865890:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23920 00000100:00100000:2.0:1787826340.865904:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346280832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.865984:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346280832:12345-0@lo:400:kworker.0 Request processed in 86us (6393us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.865990:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23920 00000100:00100000:1.0:1787826340.867942:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346280832:0@lo:400:kworker.0 00000100:00100000:2.0:1787826340.868345:0:6529:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957441 gen 2 00000004:00080000:2.0:1787826340.868351:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957439 committed transno 8589957441 00000004:00080000:2.0:1787826340.868370:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957440 committed transno 8589957441 00000004:00080000:2.0:1787826340.868385:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957441 committed transno 8589957441 00000100:00100000:2.0:1787826340.868404:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346281344:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826340.869299:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346281472:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826340.869360:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.869453:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826340.872286:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346281472 00000100:00100000:1.0:1787826340.872301:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346281472 00000100:00100000:1.0:1787826340.872361:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23921 00000100:00100000:1.0:1787826340.872370:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346281472:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826340.873477:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346281472:12345-0@lo:1000:mdt00_002.0 Request processed in 1103us (4111us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826340.873489:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23921 00000100:00100000:2.0:1787826340.876391:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346281472:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1787826340.876624:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.876635:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.876645:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.876664:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346281600:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787826340.876728:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.876817:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.880272:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346281600 00000100:00100000:0.0:1787826340.880287:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346281600 00000100:00100000:0.0:1787826340.880400:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23922 00000100:00100000:0.0:1787826340.880411:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346281600:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787826340.880464:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.880487:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b903 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.880519:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.880528:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.880537:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.880574:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.880587:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 7 pid: 24702 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.880603:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346281728:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826340.880711:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.880721:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 6 pid: 35094 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.880729:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346281856:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826340.880754:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.887787:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346281728:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826340.887804:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346281856:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826340.887844:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.887857:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826340.887861:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826340.887865:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.887875:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:2.0:1787826340.887942:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346281600:12345-0@lo:101:mdt00_002.0 Request processed in 7533us (11214us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.887950:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23922 00000100:00100000:3.0:1787826340.888523:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346281600:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787826340.888543:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.888554:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.891699:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.891753:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.891844:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332644608 00000100:00100000:3.0:1787826340.891854:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332644608 00010000:00010000:3.0:1787826340.891898:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 8 pid: 24702 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.891937:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332644736 00000100:00100000:3.0:1787826340.891941:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332644736 00010000:00010000:3.0:1787826340.891955:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 7 pid: 35094 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.891984:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22012 00000100:00100000:3.0:1787826340.891993:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332644608:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.892006:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.892013:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 8 pid: 24702 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892026:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 8 pid: 24702 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892050:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4b8fc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801854 expref: 8 pid: 24702 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892072:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826340.892078:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892088:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.892185:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332644608:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 192us (494us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.892192:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22012 00000100:00100000:3.0:1787826340.892210:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22013 00000100:00100000:3.0:1787826340.892217:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7961cdd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332644736:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.892224:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.892229:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 7 pid: 35094 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892239:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 7 pid: 35094 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892252:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b8d9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801846 expref: 7 pid: 35094 timeout: 8253 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892263:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826340.892268:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892278:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892286:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892314:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.892327:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346281984:0@lo:105: 00000100:00100000:3.0:1787826340.892364:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.892401:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb79c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.892420:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346281984 00000100:00100000:3.0:1787826340.892424:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346281984 00000100:00100000:3.0:1787826340.892473:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7897 00000100:00100000:3.0:1787826340.892479:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346281984:12345-0@lo:105: 00010000:00010000:3.0:1787826340.892489:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892500:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892508:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892515:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892522:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892534:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892555:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eec00) 00000100:00100000:3.0:1787826340.892586:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346281984:12345-0@lo:105: Request processed in 107us (221us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.892592:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7897 00000100:00100000:3.0:1787826340.892624:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346281984:0@lo:105: 00010000:00010000:3.0:1787826340.892640:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.892660:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7961cdd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332644736:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 444us (907us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.892666:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22013 00010000:00010000:3.0:1787826340.892713:0:6529:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892722:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826340.892744:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826340.892774:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892782:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.892792:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.892811:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282112:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787826340.892838:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.892855:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.893195:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346282112 00000100:00100000:0.0:1787826340.893201:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346282112 00000100:00100000:0.0:1787826340.893235:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23923 00000100:00100000:0.0:1787826340.893244:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6529:x1874663346282112:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787826340.893258:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.893273:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b918 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b911 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.893293:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b918 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b911 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.893305:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b918 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b911 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.893312:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a200) 00010000:00010000:0.0:1787826340.893316:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.893318:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826340.893355:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282112:12345-0@lo:101:mdt00_002.0 Request processed in 112us (515us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.893360:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23923 00000100:00100000:3.0:1787826340.894598:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282112:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787826340.894620:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.894689:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826340.894711:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826340.894815:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282240:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826340.894860:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.894892:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.894946:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346282240 00000100:00100000:2.0:1787826340.894952:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346282240 00000100:00100000:2.0:1787826340.895022:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23924 00000100:00100000:2.0:1787826340.895028:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282240:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826340.895097:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282240:12345-0@lo:1000:mdt00_002.0 Request processed in 68us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.895125:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23924 00000100:00100000:3.0:1787826340.895193:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282240:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826340.895225:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282368:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787826340.895245:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826340.895460:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.895591:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346282368 00000100:00100000:0.0:1787826340.895596:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346282368 00000100:00100000:0.0:1787826340.895633:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23925 00000100:00100000:0.0:1787826340.895641:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282368:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.895736:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282368:12345-0@lo:1000:mdt00_002.0 Request processed in 97us (490us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826340.895747:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23925 00000100:00100000:0.0:1787826340.895775:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282368:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.895850:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282496:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.895878:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.895893:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.895903:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346282496 00000100:00100000:0.0:1787826340.895906:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346282496 00000100:00100000:0.0:1787826340.895923:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23926 00000100:00100000:0.0:1787826340.895928:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282496:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.895992:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282496:12345-0@lo:1000:mdt00_002.0 Request processed in 63us (112us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826340.895998:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23926 00000100:00100000:0.0:1787826340.896021:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282496:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896067:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282624:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896088:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.896101:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.896135:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346282624 00000100:00100000:0.0:1787826340.896247:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346282624 00000100:00100000:0.0:1787826340.896265:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23927 00000100:00100000:0.0:1787826340.896271:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282624:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896319:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282624:12345-0@lo:1000:mdt00_002.0 Request processed in 47us (230us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.896326:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23927 00000100:00100000:0.0:1787826340.896348:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282624:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896385:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282752:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896407:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.896420:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.896430:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346282752 00000100:00100000:0.0:1787826340.896525:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346282752 00000100:00100000:0.0:1787826340.896543:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23928 00000100:00100000:0.0:1787826340.896549:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282752:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896603:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282752:12345-0@lo:1000:mdt00_002.0 Request processed in 53us (194us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.896609:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23928 00000100:00100000:0.0:1787826340.896631:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282752:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896664:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282880:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896686:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.896698:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.896709:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346282880 00000100:00100000:0.0:1787826340.896712:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346282880 00000100:00100000:0.0:1787826340.896728:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23929 00000100:00100000:0.0:1787826340.896733:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282880:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.896772:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6529:x1874663346282880:12345-0@lo:1000:mdt00_002.0 Request processed in 39us (85us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.896778:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23929 00000100:00100000:0.0:1787826340.896800:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346282880:0@lo:1000:mdt00_002.0 00000040:00080000:0.0:1787826340.898343:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1218, 2168 off1110968 00000040:00080000:0.0:1787826340.898516:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1537, 2368 off1555328 00000004:00080000:0.0:1787826340.898523:0:6529:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784c4f4600 with version 6863 00000004:00080000:2.0:1787826340.898587:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6863 rpc_version 6863 00000020:00100000:2.0:1787826340.898622:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826340.898631:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c96300 x1874663346283136/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826340.898652:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346283136:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826340.898767:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.898803:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826340.898858:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346283136 00000100:00100000:0.0:1787826340.898865:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346283136 00000100:00100000:0.0:1787826340.898938:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23930 00000100:00100000:0.0:1787826340.898947:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346283136:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826340.900186:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346283136:12345-0@lo:1000:osp_up0-1.0 Request processed in 1236us (1470us total) trans 8589957442 rc 0/0 00000100:00100000:1.0:1787826340.900201:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23930 00000100:00100000:2.0:1787826340.900366:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346283136:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826340.900523:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.900544:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.900560:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.900580:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.900587:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.900592:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.900620:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.900635:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.900833:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.900853:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346283264:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826340.900894:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.900926:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.900999:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346283264 00000100:00100000:2.0:1787826340.901005:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346283264 00000100:00100000:2.0:1787826340.901052:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23931 00000100:00100000:2.0:1787826340.901060:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6529:x1874663346283264:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787826340.901075:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826340.901091:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b934 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4b92d expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.901312:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b934 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4b92d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.901325:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b934 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b92d expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.901332:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8000) 00010000:00010000:2.0:1787826340.901335:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826340.901338:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826340.901416:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6529:x1874663346283264:12345-0@lo:101:mdt00_002.0 Request processed in 356us (521us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.901423:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23931 00000100:00100000:0.0:1787826340.901514:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346283264:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787826340.901528:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901538:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826340.901557:0:6529:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e9d747f9 reply state 00000000c782fcef transno 4294985446 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901570:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901575:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826340.901578:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826340.901580:0:6529:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826340.901587:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901595:0:6529:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826340.901607:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x197:0x0] 00000004:00010000:0.0:1787826340.901613:0:6529:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e9d747f9 reply state 00000000c782fcef transno 4294985446 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901619:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901631:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826340.901633:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826340.901634:0:6529:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787826340.901638:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e56:0x0]: returning size 4096 00000004:00080000:0.0:1787826340.901892:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826340.901900:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901906:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901910:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901926:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826340.901930:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826340.901935:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.901942:0:6529:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.901948:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826340.901955:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.901962:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826340.901967:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826340.901978:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.902003:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.902008:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826340.902030:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225825:x1874663332644480:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 42648us (42729us total) trans 4294985446 rc 0/0 00000100:00100000:0.0:1787826340.902037:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68567 00000100:00100000:2.0:1787826340.902060:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346283392:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826340.902093:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826340.902314:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.902322:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4b903 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b90a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.902335:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826340.902337:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826340.902339:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826340.902478:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346283392 00000100:00100000:2.0:1787826340.902484:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346283392 00000100:00100000:2.0:1787826340.902529:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11160 00000100:00100000:2.0:1787826340.902537:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346283392:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826340.902548:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826340.902565:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b90a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b903 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.902581:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826340.902584:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826340.902632:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346283392:12345-0@lo:103:ldlm_bl.0 Request processed in 89us (531us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.902639:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11160 00000100:00100000:3.0:1787826340.902687:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346283392:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826340.908028:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826340.908646:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332644864 00000100:00100000:1.0:1787826340.908657:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332644864 00000100:00100000:1.0:1787826340.908759:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68568 00000100:00100000:1.0:1787826340.908771:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225814:x1874663332644864:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826340.908790:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826340.908811:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b93b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080185b expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.908877:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b942 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.908890:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b942 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.908901:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b942 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826340.908946:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826340.908950:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826340.908981:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b942 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.909006:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4b93b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080185b expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.909023:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b942 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080185b expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.909035:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826340.909038:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826340.909041:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826340.909129:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225814:x1874663332644864:12345-192.168.203.56@tcp:101:bash.0 Request processed in 361us (1110us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.909606:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68568 00000100:00100000:0.0:1787826340.912028:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.912884:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332644992 00000100:00100000:2.0:1787826340.912896:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332644992 00000100:00100000:2.0:1787826340.913059:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68569 00000100:00100000:2.0:1787826340.913072:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225814:x1874663332644992:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826340.913092:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826340.913129:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4b949 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801862 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913209:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913223:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913234:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913247:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913257:0:35092:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b918 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b911 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913270:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913302:0:35092:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b918 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b911 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.913318:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb3800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346283520:0@lo:104: 00000100:00100000:2.0:1787826340.913388:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.913426:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845785cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.913455:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346283520 00000100:00100000:2.0:1787826340.913459:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346283520 00000100:00100000:2.0:1787826340.913489:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7898 00000100:00100000:2.0:1787826340.913496:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35092:x1874663346283520:12345-0@lo:104: 00010000:00010000:2.0:1787826340.913508:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913555:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826340.913561:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35092:x1874663346283520:12345-0@lo:104: Request processed in 66us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.913568:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7898 00010000:00010000:2.0:1787826340.913590:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826340.913596:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913605:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826340.913610:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826340.913624:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913647:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913654:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826340.913676:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913685:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69600/0xe21309aa81f4b911 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b918 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.913703:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826340.913705:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826340.913706:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826340.913852:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826340.913865:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346283648:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826340.913897:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826340.913947:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb3800 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346283520:0@lo:104: 00010000:00010000:2.0:1787826340.914040:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.914069:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346283648 00000100:00100000:2.0:1787826340.914073:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346283648 00010000:00010000:2.0:1787826340.914090:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b918 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b911 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826340.914140:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22014 00000100:00100000:2.0:1787826340.914148:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346283648:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826340.914158:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:1.0:1787826340.917854:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957442 is committed 00010000:00010000:2.0:1787826340.918093:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4b918 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b911 expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.918234:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a200) 00010000:00010000:2.0:1787826340.918240:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.918251:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.918257:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.918330:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826340.918433:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346283648:12345-0@lo:103:ldlm_bl.0 Request processed in 4283us (4533us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.918452:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22014 00000100:00100000:3.0:1787826340.918601:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346283648:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826340.918689:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.918704:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.918904:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346283776:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826340.918952:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.919085:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826340.919244:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346283776 00000100:00100000:3.0:1787826340.919252:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346283776 00000100:00100000:3.0:1787826340.919342:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23932 00000100:00100000:3.0:1787826340.919353:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346283776:12345-0@lo:1000:mdt00_009.0 00000100:00100000:3.0:1787826340.919840:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:35092:x1874663346283776:12345-0@lo:1000:mdt00_009.0 Request processed in 488us (886us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.919848:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23932 00000100:00100000:0.0:1787826340.919929:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-mdtlov_UUID:35092:1874663346283776:0@lo:1000:mdt00_009.0 00010000:00010000:0.0:1787826340.920051:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b95e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920065:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b95e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920074:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b95e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826340.920098:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b95e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920141:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920148:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920154:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920161:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920171:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826340.920175:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920183:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920195:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826340.920198:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.920202:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920208:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4b950 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920222:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920228:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920234:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920239:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826340.920243:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920249:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920260:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a200) 00010000:00010000:0.0:1787826340.920262:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.920265:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920271:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4b957 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920309:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4b949 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801862 expref: 9 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920323:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b95e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801862 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.920330:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f7845784400) 00010000:00010000:0.0:1787826340.920332:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826340.920334:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826340.920533:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225814:x1874663332644992:12345-192.168.203.56@tcp:101:bash.0 Request processed in 7464us (8513us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.920540:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68569 00000100:00100000:2.0:1787826340.923553:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.923638:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332645120 00000100:00100000:0.0:1787826340.923647:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332645120 00000100:00100000:0.0:1787826340.923801:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68570 00000100:00100000:0.0:1787826340.923813:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225814:x1874663332645120:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826340.923835:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.923858:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b965 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801869 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.923929:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b96c lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.923942:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b96c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.923953:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b96c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826340.924277:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e56:0x0]: returning size 4096 00000004:00200000:0.0:1787826340.924283:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826340.924326:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b965 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801869 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.924342:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b96c lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801869 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.924350:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826340.924353:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826340.924357:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826340.924603:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225814:x1874663332645120:12345-192.168.203.56@tcp:101:bash.0 Request processed in 791us (1056us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.924610:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68570 00000100:00100000:2.0:1787826340.927265:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.927332:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332645248 00000100:00100000:0.0:1787826340.927340:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332645248 00000100:00100000:0.0:1787826340.927417:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68571 00000100:00100000:0.0:1787826340.927427:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225814:x1874663332645248:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826340.927456:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.927474:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4b973 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801870 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.927530:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b97a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.927544:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b97a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.927555:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b97a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826340.927599:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x199:0x0]: returning size 4096 00000004:00200000:0.0:1787826340.927604:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826340.927637:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43000/0xe21309aa81f4b973 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801870 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.927653:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b97a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801870 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.927662:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826340.927665:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826340.927668:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826340.927730:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225814:x1874663332645248:12345-192.168.203.56@tcp:101:bash.0 Request processed in 304us (473us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.927736:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68571 00000100:00100000:1.0:1787826340.930093:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.930284:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332645376 00000100:00100000:0.0:1787826340.930293:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332645376 00000100:00100000:0.0:1787826340.930354:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68572 00000100:00100000:0.0:1787826340.930364:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:225814:x1874663332645376:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826340.930383:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.930406:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4b981 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801877 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.930476:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.930490:0:35095:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b934 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4b92d expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.930498:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.930531:0:35095:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b934 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4b92d expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.930546:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346283904:0@lo:104: 00000100:00100000:0.0:1787826340.930594:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.930736:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826340.931252:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346283904 00000100:00100000:2.0:1787826340.931259:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346283904 00000100:00100000:2.0:1787826340.931303:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7899 00000100:00100000:2.0:1787826340.931312:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346283904:12345-0@lo:104: 00010000:00010000:2.0:1787826340.931325:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826340.931452:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826340.931459:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346283904:12345-0@lo:104: Request processed in 148us (863us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826340.931465:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7899 00010000:00010000:1.0:1787826340.931524:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826340.931532:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.931540:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826340.931546:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826340.931554:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.931583:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.931590:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826340.931616:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 1/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.931624:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4b92d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4b934 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826340.931644:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826340.931663:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826340.931665:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826340.931899:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346284032:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826340.931934:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.932067:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346284032 00000100:00100000:0.0:1787826340.932073:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346284032 00010000:00010000:0.0:1787826340.932094:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b934 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4b92d expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.932208:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22015 00000100:00100000:0.0:1787826340.932217:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346284032:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826340.932228:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826340.932652:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8000) 00010000:00010000:0.0:1787826340.932658:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.932668:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.932676:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.932694:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826340.932890:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346284032:12345-0@lo:103:ldlm_bl.0 Request processed in 673us (953us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.932898:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22015 00010000:00010000:3.0:1787826340.932992:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4b934 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4b92d expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826340.933010:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346283904:0@lo:104: 00010000:00010000:3.0:1787826340.933032:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.933041:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.933047:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826340.933133:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x197:0x0]: returning size 4096 00000004:00200000:3.0:1787826340.933138:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826340.933185:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4b981 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801877 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.933201:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801877 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.933208:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8000) 00010000:00010000:3.0:1787826340.933211:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826340.933213:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:3.0:1787826340.933263:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225814:x1874663332645376:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2901us (3179us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.933269:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68572 00000100:00100000:0.0:1787826340.936582:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.936657:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332645632 00000100:00100000:3.0:1787826340.936668:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332645632 00000100:00100000:3.0:1787826340.936735:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68573 00000100:00100000:3.0:1787826340.936747:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225814:x1874663332645632:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826340.936766:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826340.936769:0:24703:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826340.936796:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4b988 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801877 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.936816:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8000) 00010000:00010000:3.0:1787826340.936828:0:24703:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826340.936844:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b98f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080187e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.936977:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.936989:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.936999:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.937012:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.937020:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.937027:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826340.937610:0:24703:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x522:0x0], index 1: 0 00010000:00010000:3.0:1787826340.938039:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b9a4 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938051:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b9a4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938060:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b9a4 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938095:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938102:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938391:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938398:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938404:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826340.938409:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938416:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938427:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f79460a2800) 00010000:00010000:3.0:1787826340.938431:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826340.938435:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938456:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b996 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826340.938471:0:24703:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bef3987a reply state 000000007ffb2b9a transno 8589957443 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938482:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938488:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f79460a2900) 00010000:00010000:3.0:1787826340.938490:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826340.938492:0:24703:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826340.938530:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4b98f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080187e expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938549:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b9a4 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080187e expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.938557:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f79460a2500) 00010000:00010000:3.0:1787826340.938560:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826340.938562:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826340.938664:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225814:x1874663332645632:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1920us (2092us total) trans 8589957443 rc 0/0 00000100:00100000:3.0:1787826340.938673:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68573 00000100:00100000:2.0:1787826340.938917:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b61ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346284032:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826340.943253:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.943358:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332645760 00000100:00100000:0.0:1787826340.943368:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332645760 00000100:00100000:0.0:1787826340.943431:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30860 00000100:00100000:0.0:1787826340.943449:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225814:x1874663332645760:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826340.943468:0:35352:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.943489:0:35352:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801885 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826340.943516:0:35352:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x522:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787826340.943542:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=8153s v=(0 0) 00010000:00010000:0.0:1787826340.943547:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826340.943556:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 7 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.943579:0:35352:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 7 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.943592:0:35352:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826340.943647:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225814:x1874663332645760:12345-192.168.203.56@tcp:101:bash.0 Request processed in 207us (403us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826340.943654:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30860 00000100:00100000:1.0:1787826340.946214:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826340.946385:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332646016 00000100:00100000:2.0:1787826340.946396:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332646016 00000100:00100000:2.0:1787826340.946495:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9373 00000100:00100000:2.0:1787826340.946506:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943030850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225814:x1874663332646016:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826340.946798:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943030850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225814:x1874663332646016:12345-192.168.203.56@tcp:35:bash.0 Request processed in 293us (593us total) trans 8589957444 rc 0/0 00000100:00100000:2.0:1787826340.946805:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9373 00000100:00100000:3.0:1787826340.950918:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826340.951090:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332646272 00000100:00100000:1.0:1787826340.951100:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332646272 00000100:00100000:0.0:1787826340.951368:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68574 00000100:00100000:0.0:1787826340.951389:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225814:x1874663332646272:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826340.951410:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826340.951413:0:6529:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826340.951453:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4b97a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801870 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.951474:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826340.951479:0:6529:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826340.951493:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b9b9 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080188c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.951618:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.951631:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.951641:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.951657:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.951665:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.951671:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826340.951811:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346284160:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826340.951869:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826340.951903:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826340.952076:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346284160 00000100:00100000:1.0:1787826340.952082:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346284160 00000100:00100000:1.0:1787826340.952153:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23933 00000100:00100000:1.0:1787826340.952162:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346284160:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826340.952247:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6529:x1874663346284160:12345-0@lo:1000:mdt00_002.0 Request processed in 86us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826340.952255:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23933 00000100:00100000:0.0:1787826340.952458:0:6529:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957442 gen 2 00000004:00080000:0.0:1787826340.952464:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957442 committed transno 8589957442 00000100:00100000:0.0:1787826340.952494:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6529:1874663346284160:0@lo:1000:mdt00_002.0 00000004:00080000:0.0:1787826340.953388:0:6529:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4cf:0x0], index 0: 0 00010000:00010000:0.0:1787826340.954607:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b9ce lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.954626:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b9ce lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.954920:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b9ce lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.954964:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.954971:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.954978:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.954985:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.954990:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826340.954995:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.955002:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.955016:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826340.955019:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826340.955024:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.955030:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4b9c0 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826340.955046:0:6529:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005e24692a reply state 00000000e8b82b9a transno 4294985447 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.955056:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.955062:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x4ec94fd6 (ffff9f7845784100) 00010000:00010000:0.0:1787826340.955064:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826340.955067:0:6529:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826340.955142:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4b9b9 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080188c expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.955158:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b9ce lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080188c expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826340.955166:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7845784e00) 00010000:00010000:0.0:1787826340.955168:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826340.955220:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826340.955361:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225814:x1874663332646272:12345-192.168.203.56@tcp:101:bash.0 Request processed in 3983us (4453us total) trans 4294985447 rc 0/0 00000100:00100000:0.0:1787826340.955370:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68574 00000100:00100000:1.0:1787826340.959801:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.959893:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332646400 00000100:00100000:3.0:1787826340.959903:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332646400 00000100:00100000:3.0:1787826340.959976:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30861 00000100:00100000:3.0:1787826340.959988:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:225814:x1874663332646400:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826340.960005:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826340.960025:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801893 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826340.960051:0:35096:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4cf:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1787826340.960071:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cf:0x0].0x0 since=8153s v=(0 0) 00010000:00010000:3.0:1787826340.960076:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826340.960086:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.960134:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 6 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.960150:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826340.960213:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225814:x1874663332646400:12345-192.168.203.56@tcp:101:bash.0 Request processed in 227us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.960220:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30861 00000100:00100000:1.0:1787826340.963089:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.963222:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332646656 00000100:00100000:3.0:1787826340.963230:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332646656 00000100:00100000:3.0:1787826340.963290:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9374 00000100:00100000:3.0:1787826340.963300:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225814:x1874663332646656:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826340.963620:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225814:x1874663332646656:12345-192.168.203.56@tcp:35:bash.0 Request processed in 320us (537us total) trans 4294985448 rc 0/0 00000100:00100000:3.0:1787826340.963628:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9374 00000100:00100000:1.0:1787826340.983948:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.984182:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332646784 00000100:00100000:3.0:1787826340.984192:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332646784 00000100:00100000:3.0:1787826340.984466:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11161 00000100:00100000:3.0:1787826340.984481:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:34880:x1874663332646784:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.984497:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826340.984523:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4b942 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080185b expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.984541:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826340.984552:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b9a4 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080187e expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.984561:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f79460a2500) 00010000:00010000:3.0:1787826340.984575:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4b95e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801862 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.984585:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f7845784400) 00010000:00010000:3.0:1787826340.984592:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.984753:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:34880:x1874663332646784:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 276us (813us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.984762:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11161 00000100:00100000:0.0:1787826340.988165:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826340.988206:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826340.988350:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332646912 00000100:00100000:3.0:1787826340.988360:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332646912 00000100:00100000:3.0:1787826340.988430:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332647040 00000100:00100000:3.0:1787826340.988433:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332647040 00000100:00100000:3.0:1787826340.988468:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11162 00000100:00100000:3.0:1787826340.988479:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:6167:x1874663332646912:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.988496:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826340.988521:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4b7c8 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dbe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080180e expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.988541:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbe:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:3.0:1787826340.988550:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.988593:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:6167:x1874663332646912:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 117us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.988600:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11162 00000100:00100000:3.0:1787826340.988620:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11163 00000100:00100000:3.0:1787826340.988627:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:34879:x1874663332647040:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826340.988636:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826340.988650:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4b9ce lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080188c expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.988661:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7845784e00) 00010000:00010000:3.0:1787826340.988674:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4b96c lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801869 expref: 12 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.988683:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:3.0:1787826340.988690:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4b727 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e54:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017eb expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826340.988699:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e54:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:3.0:1787826340.988705:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826340.988914:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:34879:x1874663332647040:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 287us (707us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826340.988921:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11163 00010000:00010000:1.0:1787826341.036365:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826341.356697:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346284288:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826341.356799:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826341.362316:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346284288 00000100:00100000:3.0:1787826341.362331:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346284288 00000100:00100000:3.0:1787826341.362547:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6247 00000100:00100000:3.0:1787826341.362558:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346284288:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826341.362592:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914711040 avail 3683188736 00000100:00100000:3.0:1787826341.365170:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346284288:12345-0@lo:13:osp-pre-1-1.0 Request processed in 2604us (8360us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826341.365184:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6247 00000100:00100000:1.0:1787826341.365470:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346284288:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826341.869034:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346284416:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826341.869141:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826341.871035:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346284416 00000100:00100000:1.0:1787826341.871047:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346284416 00000100:00100000:1.0:1787826341.871100:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6248 00000100:00100000:1.0:1787826341.871128:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3655:x1874663346284416:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826341.871156:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826341.871231:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3655:x1874663346284416:12345-0@lo:13:osp-pre-0-1.0 Request processed in 104us (2089us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826341.871238:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6248 00000100:00100000:1.0:1787826341.871284:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346284416:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826341.871564:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346284672:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826341.871604:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826341.871630:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826341.871636:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346284544:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826341.871655:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826341.871675:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346284672 00000100:00100000:2.0:1787826341.871678:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346284672 00000100:00100000:2.0:1787826341.871698:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346284544 00000100:00100000:2.0:1787826341.871700:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346284544 00000100:00100000:2.0:1787826341.871716:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6249 00000100:00100000:2.0:1787826341.871721:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346284672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826341.871732:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914711040 avail 3683188736 00000100:00100000:2.0:1787826341.871761:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346284672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 39us (157us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826341.871767:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6249 00000100:00100000:2.0:1787826341.871779:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6250 00000100:00100000:2.0:1787826341.871785:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346284544:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826341.871793:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826341.871811:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346284544:12345-0@lo:13:osp-pre-0-0.0 Request processed in 27us (156us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826341.871816:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6250 00000100:00100000:2.0:1787826341.871840:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346284544:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826341.871850:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346284672:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826341.871873:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346284800:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826341.871898:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1787826341.872018:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826341.872032:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346284800 00000100:00100000:1.0:1787826341.872035:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346284800 00000100:00100000:1.0:1787826341.872061:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23934 00000100:00100000:1.0:1787826341.872066:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346284800:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826341.872088:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442779136 avail 1296539648 00000100:00100000:1.0:1787826341.872137:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346284800:12345-0@lo:41:osp-pre-0-1.0 Request processed in 70us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826341.872143:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23934 00000100:00100000:2.0:1787826341.872217:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346284800:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826341.872228:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787826341.872233:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346284928:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826341.872254:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826341.873068:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346284928 00000100:00100000:1.0:1787826341.873075:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346284928 00000100:00100000:1.0:1787826341.873136:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23935 00000100:00100000:1.0:1787826341.873145:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346284928:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826341.873168:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443381248 avail 1297141760 00000100:00100000:1.0:1787826341.873226:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346284928:12345-0@lo:41:osp-pre-1-0.0 Request processed in 82us (970us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826341.873232:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23935 00000100:00100000:1.0:1787826341.873273:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346284928:0@lo:41:osp-pre-1-0.0 00010000:00010000:3.0:1787826342.061251:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826342.463572:0:166832:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:0.0:1787826342.463617:0:166832:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000001:00080000:0.0:1787826342.896097:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973856 is committed 00000001:00080000:0.0:1787826342.896145:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973855 is committed 00010000:00010000:3.0:1787826342.904496:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4b869 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826342.904518:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4b869 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826342.904569:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b869 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826342.904576:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826342.904644:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346285056:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826342.904739:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826342.904824:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecea00/0xe21309aa81f4b862 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4b869 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826342.904848:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985448 is committed 00000020:00080000:3.0:1787826342.904867:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784c4f4300 refcount 2 committed 0 result 0 batchid 3433 00000020:00080000:3.0:1787826342.904871:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784a925000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784c4f4600 00000020:00080000:3.0:1787826342.904875:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1537 00000020:00080000:3.0:1787826342.904877:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784a925700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78472a2200 00000020:00080000:3.0:1787826342.904880:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1218 00000001:00080000:3.0:1787826342.904892:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985445 is committed 00000020:00080000:3.0:1787826342.904897:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784b482a80 refcount 2 committed 0 result 0 batchid 3432 00000020:00080000:3.0:1787826342.904900:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784421e100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784b4823c0 00000020:00080000:3.0:1787826342.904903:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1536 00000020:00080000:3.0:1787826342.904905:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784421e600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784b76a800 00000020:00080000:3.0:1787826342.904907:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1217 00000004:00080000:3.0:1787826342.904914:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 406 committed 00000020:00080000:3.0:1787826342.904924:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784afb73c0 refcount 2 committed 0 result 0 batchid 3431 00000020:00080000:3.0:1787826342.904928:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784afb79c0 00000020:00080000:3.0:1787826342.904930:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1535 00000020:00080000:3.0:1787826342.904932:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259800 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78472a3a00 00000020:00080000:3.0:1787826342.904935:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1216 00000020:00080000:3.0:1787826342.904943:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78484a4240 refcount 2 committed 0 result 0 batchid 3430 00000020:00080000:3.0:1787826342.904946:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7851d813c0 00000020:00080000:3.0:1787826342.904961:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1534 00000020:00080000:3.0:1787826342.904963:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784b76ac00 00000020:00080000:3.0:1787826342.904966:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1215 00000001:00080000:3.0:1787826342.904971:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985439 is committed 00000020:00080000:3.0:1787826342.904982:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3429 batchid updated 00000004:00080000:3.0:1787826342.904988:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 367 committed 00000020:00080000:3.0:1787826342.905012:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3433 committed batchid 3429 00010000:00010000:0.0:1787826342.907493:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907511:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907521:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826342.907521:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346285056 00010000:00010000:0.0:1787826342.907531:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00000100:00100000:1.0:1787826342.907534:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346285056 00010000:00010000:0.0:1787826342.907537:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907547:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907565:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826342.907568:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826342.907572:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907579:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4b91f lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907605:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826342.907609:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11164 00010000:00010000:0.0:1787826342.907618:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826342.907624:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b470c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346285056:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787826342.907628:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907636:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826342.907639:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826342.907640:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907650:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907661:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826342.907665:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826342.907665:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4b869 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x196:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4b862 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907670:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907688:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4b926 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826342.907690:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x196:0x0].0x5a734ce9 (ffff9f7967ac8f00) 00010000:00010000:1.0:1787826342.907705:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826342.907715:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907723:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907729:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907734:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826342.907738:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907745:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907753:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x4ec94fd6 (ffff9f7845784100) 00010000:00010000:0.0:1787826342.907756:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826342.907760:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.907766:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4b9c7 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826342.907785:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b470c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346285056:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 164us (3045us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826342.907793:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11164 00000100:00100000:0.0:1787826342.907859:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346285056:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826342.907921:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346285184:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1787826342.907954:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826342.908001:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346285184 00000100:00100000:0.0:1787826342.908007:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346285184 00000100:00100000:0.0:1787826342.908054:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30862 00000100:00080000:1.0:1787826342.908057:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826342.908064:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346285184:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1787826342.908089:0:8759:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1228 count 1 00010000:00010000:0.0:1787826342.908151:0:8759:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.908174:0:8759:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.908184:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4cc:0x0].0x0 since=9s v=(0 0) 00010000:00010000:0.0:1787826342.908189:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826342.908194:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.908225:0:8759:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.908238:0:8759:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 6 pid: 17826 timeout: 8255 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826342.908265:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8759:1874663346285312:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826342.908510:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784daae000 going to sleep for 11 seconds 00000004:00080000:3.0:1787826342.910413:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784afb7540 with version 6864 00000020:00080000:3.0:1787826342.910441:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3426 cancel update log [0x200000401:0x3:0x0].1530: rc = 0 00000020:00080000:3.0:1787826342.910520:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3426 cancel update log [0x240000400:0x3:0x0].1211: rc = 0 00000004:00080000:3.0:1787826342.910552:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784549ae40 with version 6865 00000020:00080000:3.0:1787826342.910556:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3427 cancel update log [0x200000401:0x3:0x0].1531: rc = 0 00000020:00080000:3.0:1787826342.910574:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3427 cancel update log [0x240000400:0x3:0x0].1212: rc = 0 00000004:00080000:3.0:1787826342.910589:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78434e4540 with version 6866 00000020:00080000:3.0:1787826342.910591:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3428 cancel update log [0x200000401:0x3:0x0].1532: rc = 0 00000020:00080000:3.0:1787826342.910604:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3428 cancel update log [0x240000400:0x3:0x0].1213: rc = 0 00000004:00080000:3.0:1787826342.910618:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78434e4a80 with version 6867 00000020:00080000:3.0:1787826342.910620:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3429 cancel update log [0x200000401:0x3:0x0].1533: rc = 0 00000020:00080000:3.0:1787826342.910634:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3429 cancel update log [0x240000400:0x3:0x0].1214: rc = 0 00000020:00080000:3.0:1787826342.910643:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3429 00000004:00080000:1.0:1787826342.910761:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6864 rpc_version 6864 00000020:00100000:1.0:1787826342.910796:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826342.910807:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b75a300 x1874663346285568/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826342.910831:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346285568:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826342.910899:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826342.910941:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78474cac00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826342.912157:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346285568 00000100:00100000:3.0:1787826342.912169:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346285568 00000100:00100000:3.0:1787826342.912300:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23936 00000100:00100000:3.0:1787826342.912312:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346285568:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826342.912857:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346285568:12345-0@lo:1000:osp_up0-1.0 Request processed in 547us (1956us total) trans 8589957445 rc 0/0 00000100:00100000:3.0:1787826342.912865:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23936 00000100:00100000:0.0:1787826342.912915:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75a300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346285568:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826342.912932:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6865 rpc_version 6865 00000020:00100000:0.0:1787826342.912955:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826342.912962:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78433ef100 x1874663346285824/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826342.912983:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346285824:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826342.913021:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826342.913046:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784daae240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826342.914461:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346285824 00000100:00100000:3.0:1787826342.914473:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346285824 00000100:00100000:3.0:1787826342.914592:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23937 00000100:00100000:3.0:1787826342.914604:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346285824:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826342.915535:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826342.915561:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332647168 00000100:00100000:0.0:1787826342.915566:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332647168 00010000:00010000:0.0:1787826342.915604:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 7 pid: 17826 timeout: 8255 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826342.915634:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22016 00000100:00100000:0.0:1787826342.915641:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332647168:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826342.915653:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826342.915658:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 7 pid: 17826 timeout: 8255 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.915671:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 7 pid: 17826 timeout: 8255 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.915708:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826342.915714:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826342.915720:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.915730:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.915757:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826342.915796:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332647168:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 154us (266us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826342.915802:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22016 00010000:00010000:0.0:1787826342.915933:0:8759:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f7d23212/0xe21309aa81f4b67f lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e87608017ac expref: 6 pid: 17826 timeout: 8255 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826342.915948:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8759:1874663346285312:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826342.915967:0:8759:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.915976:0:8759:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.915982:0:8759:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.915991:0:8759:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.915998:0:8759:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.916016:0:8759:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826342.916212:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346285184:12345-0@lo:6:osp-syn-1-1.0 Request processed in 8150us (8256us total) trans 4294976165 rc 0/0 00000100:00100000:0.0:1787826342.916221:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30862 00010000:00010000:0.0:1787826342.916243:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826342.916250:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.916258:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826342.916263:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.916272:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.916285:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.916293:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a433ba8e/0xe21309aa81f4b9e3 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4cc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826342.916326:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826342.916328:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826342.916331:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826342.916357:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:0.0:1787826342.916403:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784b75ad80/2, rc 0, transno 4294976165 00000100:00100000:0.0:1787826342.916410:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346285184:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1787826342.916496:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346285824:12345-0@lo:1000:osp_up0-1.0 Request processed in 1892us (3472us total) trans 8589957446 rc 0/0 00000100:00100000:3.0:1787826342.916506:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23937 00000100:00100000:1.0:1787826342.923512:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346285824:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826342.923532:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6866 rpc_version 6866 00000020:00100000:1.0:1787826342.923559:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826342.923566:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b75aa00 x1874663346286080/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826342.923596:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346286080:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826342.923662:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826342.923687:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784adfccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826342.923878:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346286080 00000100:00100000:3.0:1787826342.923884:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346286080 00000100:00100000:3.0:1787826342.923941:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23938 00000100:00100000:3.0:1787826342.923950:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346286080:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826342.924153:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346286080:12345-0@lo:1000:osp_up0-1.0 Request processed in 203us (489us total) trans 8589957447 rc 0/0 00000100:00100000:3.0:1787826342.924164:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23938 00000100:00100000:1.0:1787826342.924340:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346286080:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826342.924352:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6867 rpc_version 6867 00000020:00100000:1.0:1787826342.924366:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826342.924371:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b758000 x1874663346286336/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826342.924384:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b758000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346286336:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826342.924412:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826342.924485:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826342.924520:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346286336 00000100:00100000:3.0:1787826342.924525:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346286336 00000100:00100000:3.0:1787826342.924557:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23939 00000100:00100000:3.0:1787826342.924564:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346286336:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826342.924683:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346286336:12345-0@lo:1000:osp_up0-1.0 Request processed in 120us (270us total) trans 8589957448 rc 0/0 00000100:00100000:3.0:1787826342.924690:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23939 00000100:00100000:1.0:1787826342.924737:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b758000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346286336:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826343.084413:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826344.114710:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826344.725530:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826344.725604:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332647296 00000100:00100000:0.0:1787826344.725614:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332647296 00000100:00100000:0.0:1787826344.725696:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68575 00000100:00100000:0.0:1787826344.725707:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225995:x1874663332647296:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826344.725726:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826344.725748:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4b9ea lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080189a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826344.725813:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826344.725828:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826344.725839:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826344.725889:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826344.725893:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826344.725915:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826344.725939:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4b9ea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826344.725955:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826344.725964:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826344.725967:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826344.725970:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826344.726028:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225995:x1874663332647296:12345-192.168.203.56@tcp:101:cat.0 Request processed in 323us (507us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826344.726035:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68575 00000100:00080000:1.0:1787826344.726830:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826344.735055:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826344.735144:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332647424 00000100:00100000:3.0:1787826344.735155:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332647424 00000100:00100000:3.0:1787826344.735214:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68576 00000100:00100000:3.0:1787826344.735225:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225995:x1874663332647424:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826344.735244:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826344.735265:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b9f8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018a1 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735329:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735343:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735354:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735369:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735377:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735383:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735472:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ba0d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735480:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ba0d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735486:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ba0d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826344.735509:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ba0d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735523:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735529:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735539:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735545:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735550:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826344.735555:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735562:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735573:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826344.735576:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826344.735581:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735587:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4b9ff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735601:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735607:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735613:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735617:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826344.735620:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735626:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735632:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7843326100) 00010000:00010000:3.0:1787826344.735634:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826344.735637:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735643:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4ba06 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735674:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4b9f8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018a1 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735912:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ba0d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018a1 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826344.735920:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f7843326b00) 00010000:00010000:3.0:1787826344.735922:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826344.735924:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826344.736024:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225995:x1874663332647424:12345-192.168.203.56@tcp:101:cat.0 Request processed in 800us (980us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826344.736033:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68576 00000100:00100000:0.0:1787826344.743952:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826344.744021:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332647552 00000100:00100000:1.0:1787826344.744031:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332647552 00000100:00100000:1.0:1787826344.744130:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68577 00000100:00100000:1.0:1787826344.744143:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:225995:x1874663332647552:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826344.744168:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826344.744191:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ba14 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018a8 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826344.744249:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4ba1b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826344.744262:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4ba1b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826344.744274:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4ba1b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826344.744369:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e56:0x0]: returning size 4096 00000004:00200000:1.0:1787826344.744374:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826344.744409:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4ba14 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018a8 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826344.744440:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4ba1b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018a8 expref: 6 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826344.744450:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794385a400) 00010000:00010000:1.0:1787826344.744453:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826344.744456:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826344.744514:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225995:x1874663332647552:12345-192.168.203.56@tcp:101:cat.0 Request processed in 375us (581us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826344.744522:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68577 00000100:00080000:0.0:1787826344.744810:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826344.752823:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826344.752897:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332647680 00000100:00100000:2.0:1787826344.752908:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332647680 00000100:00100000:2.0:1787826344.752977:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68578 00000100:00100000:2.0:1787826344.752989:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225995:x1874663332647680:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826344.753009:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826344.753033:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4ba22 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018af expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.753100:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.753150:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.753163:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826344.753207:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x199:0x0]: returning size 4096 00000004:00200000:2.0:1787826344.753211:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826344.753246:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4ba22 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.753264:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.753274:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:2.0:1787826344.753277:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826344.753280:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826344.753338:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:225995:x1874663332647680:12345-192.168.203.56@tcp:101:cat.0 Request processed in 352us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826344.753345:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68578 00000100:00100000:0.0:1787826344.757835:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826344.757912:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332647808 00000100:00100000:2.0:1787826344.757921:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332647808 00000100:00100000:2.0:1787826344.757982:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68579 00000100:00100000:2.0:1787826344.757993:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:225995:x1874663332647808:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826344.758016:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826344.758050:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ba30 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018b6 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.758145:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.758158:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.758168:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826344.758210:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x197:0x0]: returning size 4096 00000004:00200000:2.0:1787826344.758215:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826344.758253:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ba30 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.758272:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.758280:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8b00) 00010000:00010000:2.0:1787826344.758283:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826344.758287:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826344.758345:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225995:x1874663332647808:12345-192.168.203.56@tcp:101:cat.0 Request processed in 353us (519us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826344.758353:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68579 00000100:00100000:0.0:1787826344.771298:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826344.775193:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332648064 00000100:00100000:2.0:1787826344.775207:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332648064 00000100:00100000:2.0:1787826344.778971:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68580 00000100:00100000:2.0:1787826344.778996:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225995:x1874663332648064:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826344.779018:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826344.779042:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ba3e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018bd expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.779141:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.779156:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.779167:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.779179:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.779187:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826344.779194:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:2.0:1787826344.780568:0:35093:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00010000:00010000:3.0:1787826345.133294:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826345.180217:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.180262:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826345.180606:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332648192 00000100:00100000:2.0:1787826345.180616:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332648192 00000100:00100000:0.0:1787826345.180740:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.180770:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.180796:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.180817:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.180835:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.180853:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.180876:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.180897:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332648448 00000100:00100000:0.0:1787826345.180901:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332648448 00000100:00100000:0.0:1787826345.180943:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332648960 00000100:00100000:0.0:1787826345.180946:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332648960 00000100:00100000:0.0:1787826345.180971:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332648320 00000100:00100000:0.0:1787826345.180974:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332648320 00000100:00100000:0.0:1787826345.180989:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332648832 00000100:00100000:0.0:1787826345.180991:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332648832 00000100:00100000:0.0:1787826345.181014:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14344 00000100:00100000:0.0:1787826345.181024:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2395:x1874663332648448:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826345.181165:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2234 00000100:00100000:2.0:1787826345.181173:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79747d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332648192:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826345.181249:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79747d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332648192:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 76us (1051us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.181256:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2234 00000100:00100000:2.0:1787826345.181294:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14345 00000100:00100000:2.0:1787826345.181302:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332648960:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826345.181332:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332648960:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 31us (561us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.181338:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14345 00000100:00100000:2.0:1787826345.181355:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14346 00000100:00100000:2.0:1787826345.181361:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2395:x1874663332648320:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826345.181384:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2395:x1874663332648320:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (567us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.181390:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14346 00000100:00100000:2.0:1787826345.181402:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14347 00000100:00100000:2.0:1787826345.181409:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332648832:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826345.181478:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332648832:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 69us (623us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.181484:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14347 00000100:00100000:2.0:1787826345.181504:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332648704 00000100:00100000:2.0:1787826345.181508:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332648704 00000100:00100000:2.0:1787826345.181534:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332649216 00000100:00100000:2.0:1787826345.181536:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332649216 00000100:00100000:2.0:1787826345.181552:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332648576 00000100:00100000:2.0:1787826345.181555:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332648576 00000100:00100000:2.0:1787826345.181569:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332649088 00000100:00100000:2.0:1787826345.181572:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332649088 00000100:00100000:2.0:1787826345.181589:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8855 00000100:00100000:2.0:1787826345.181596:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332648704:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826345.181620:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332648704:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (880us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.181626:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8855 00000100:00080000:1.0:1787826345.181871:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826345.181876:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826345.181879:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787826345.181881:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826345.181903:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8856 00000100:00100000:1.0:1787826345.181913:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332649216:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826345.181942:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332649216:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 33us (1145us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.181949:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8856 00000100:00100000:1.0:1787826345.181964:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8857 00000100:00100000:1.0:1787826345.181971:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332648576:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826345.181995:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332648576:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (1158us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.182001:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8857 00000100:00100000:1.0:1787826345.182013:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8858 00000100:00100000:1.0:1787826345.182019:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332649088:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826345.182735:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332649088:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (1164us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.182750:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8858 00000100:00100000:0.0:1787826345.183453:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2395:x1874663332648448:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 2426us (3185us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.183464:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14344 00000100:00100000:0.0:1787826345.241387:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826345.241910:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332649344 00000100:00100000:2.0:1787826345.241921:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332649344 00000100:00100000:2.0:1787826345.241987:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68581 00000100:00100000:2.0:1787826345.241999:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:225997:x1874663332649344:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826345.242021:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826345.242607:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ba53 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018c4 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.242874:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.242889:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.242902:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826345.242953:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826345.242958:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826345.242981:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.243010:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ba53 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.243034:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.244272:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826345.244280:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826345.244285:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826345.244444:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225997:x1874663332649344:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2436us (3055us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.244454:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68581 00000100:00100000:1.0:1787826345.263188:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826345.263302:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332649472 00000100:00100000:2.0:1787826345.263312:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332649472 00000100:00100000:2.0:1787826345.263373:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68582 00000100:00100000:2.0:1787826345.263384:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:225997:x1874663332649472:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826345.263404:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826345.263446:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ba61 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018cb expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263516:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263529:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263543:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263560:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263567:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263573:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263654:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263664:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263671:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826345.263706:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263721:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263727:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263733:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263739:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263744:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826345.263749:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263757:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263769:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826345.263772:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826345.263777:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263783:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4ba68 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263797:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263804:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263810:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263815:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826345.263817:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263823:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263829:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8000) 00010000:00010000:2.0:1787826345.263832:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826345.263835:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263844:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba6f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263875:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ba61 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263888:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.263895:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f7843326b00) 00010000:00010000:2.0:1787826345.263897:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826345.263900:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:2.0:1787826345.263967:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225997:x1874663332649472:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 584us (789us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.263974:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68582 00000100:00100000:3.0:1787826345.270998:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826345.271165:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332649600 00000100:00100000:2.0:1787826345.271176:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332649600 00000100:00100000:2.0:1787826345.271464:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68583 00000100:00100000:2.0:1787826345.271476:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:225997:x1874663332649600:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826345.271499:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826345.271521:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4ba7d lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018d2 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.271581:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba84 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.271593:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba84 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.271604:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba84 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826345.271719:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e56:0x0]: returning size 4096 00000004:00200000:2.0:1787826345.271725:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826345.271765:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4ba7d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018d2 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.271786:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba84 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018d2 expref: 6 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.271794:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794385a400) 00010000:00010000:2.0:1787826345.271797:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826345.271801:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826345.271869:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225997:x1874663332649600:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 396us (881us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.271876:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68583 00000100:00100000:3.0:1787826345.278058:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826345.278384:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332649728 00000100:00100000:1.0:1787826345.278394:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332649728 00000100:00100000:1.0:1787826345.278479:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68584 00000100:00100000:1.0:1787826345.278490:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:225997:x1874663332649728:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826345.278510:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826345.278532:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ba8b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018d9 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.278598:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.278611:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.278626:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.278642:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.278649:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.278656:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279006:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4baa0 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279015:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4baa0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279022:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4baa0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826345.279265:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e57:0x0]: returning size 6 00000004:00200000:1.0:1787826345.279269:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826345.279291:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4baa0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279303:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4baa0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279311:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279321:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279326:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279333:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279338:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826345.279343:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279350:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279361:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:1.0:1787826345.279364:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826345.279368:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279375:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ba92 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279390:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279396:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279402:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279407:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826345.279410:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.279426:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.280002:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x4ec94fd6 (ffff9f794385a100) 00010000:00010000:1.0:1787826345.280007:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826345.280011:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.280018:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4ba99 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.280186:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ba8b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018d9 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.281144:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4baa0 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018d9 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.281156:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f794385a700) 00010000:00010000:1.0:1787826345.281158:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826345.281161:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:1.0:1787826345.281551:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:225997:x1874663332649728:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 3060us (3507us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.281562:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68584 00000100:00100000:3.0:1787826345.287754:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.296450:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332649856 00000100:00100000:0.0:1787826345.296468:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332649856 00000100:00100000:0.0:1787826345.296556:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68585 00000100:00100000:0.0:1787826345.296568:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:225997:x1874663332649856:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826345.296587:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826345.296609:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4baa7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018e0 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296677:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 3/1,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296836:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296848:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296861:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296869:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296875:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296979:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4babc lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296987:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4babc lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.296994:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4babc lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826345.297265:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbf:0x0]: returning size 6 00000004:00200000:0.0:1787826345.297270:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826345.297295:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4babc lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297307:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4babc lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297418:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297425:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297477:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297486:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297493:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826345.297498:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297505:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297519:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8b00) 00010000:00010000:0.0:1787826345.297522:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826345.297527:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 3/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297534:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4baae lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297548:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297554:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297560:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297565:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826345.297568:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297574:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297581:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f79460a2900) 00010000:00010000:0.0:1787826345.297584:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826345.297587:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297592:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4bab5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297622:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4baa7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018e0 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297636:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4babc lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018e0 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.297642:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f7845784c00) 00010000:00010000:0.0:1787826345.297645:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826345.297647:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826345.297821:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225997:x1874663332649856:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1254us (10076us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.297830:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68585 00000100:00100000:0.0:1787826345.301828:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.301869:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.301897:0:6542:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332650240 00000100:00100000:0.0:1787826345.301906:0:6542:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332650240 00000100:00100000:0.0:1787826345.301975:0:6542:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1234 00000100:00100000:0.0:1787826345.301987:0:6542:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:225997:x1874663332650240:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:0.0:1787826345.302051:0:6542:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826345.302080:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4babc lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018e0 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.302100:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f7845784c00) 00010000:00010000:0.0:1787826345.302134:0:6542:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826345.302251:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286464:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.302304:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.302353:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d45480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.302372:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332649984 00000100:00100000:0.0:1787826345.302375:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332649984 00000100:00100000:0.0:1787826345.302425:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11165 00000100:00100000:0.0:1787826345.302445:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332649984:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:0.0:1787826345.302457:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826345.302470:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4baa0 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018d9 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.302482:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f794385a700) 00010000:00010000:0.0:1787826345.302489:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826345.302532:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332649984:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 99us (661us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.302538:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11165 00000100:00100000:0.0:1787826345.302566:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346286464 00000100:00100000:0.0:1787826345.302569:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346286464 00000100:00100000:0.0:1787826345.302593:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23940 00000100:00100000:0.0:1787826345.302599:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286464:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.302663:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286464:12345-0@lo:1000:mdt_io00_002.0 Request processed in 64us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.302669:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23940 00000100:00100000:0.0:1787826345.302703:0:6542:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985439 gen 2 00000004:00080000:0.0:1787826345.302709:0:6542:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985438 committed transno 4294985439 00000020:00080000:0.0:1787826345.302714:0:6542:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7850751600 refcount 2 committed 0 result 0 batchid 4294979086 00000020:00080000:0.0:1787826345.302718:0:6542:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a3b0f00 00000020:00080000:0.0:1787826345.302721:0:6542:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].943 00000020:00080000:0.0:1787826345.302724:0:6542:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7851259000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78472a3a00 00000020:00080000:0.0:1787826345.302726:0:6542:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].939 00000004:00080000:0.0:1787826345.302749:0:6542:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985439 committed transno 4294985439 00000100:00100000:0.0:1787826345.302762:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286464:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.302811:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286592:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.302835:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.302853:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d45480 going to sleep for 11 seconds 00000020:00080000:0.0:1787826345.302864:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979086 committed batchid 4294979085 00000020:00080000:0.0:1787826345.302924:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979085 00000100:00100000:0.0:1787826345.302937:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346286592 00000100:00100000:0.0:1787826345.302941:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346286592 00000100:00100000:0.0:1787826345.302964:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23941 00000100:00100000:0.0:1787826345.302971:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286592:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.303032:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286592:12345-0@lo:1000:mdt_io00_002.0 Request processed in 60us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.303038:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23941 00000100:00100000:0.0:1787826345.303063:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286592:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826345.303086:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x199:0x0] 00000100:00100000:0.0:1787826345.303133:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286720:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.303156:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.303171:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d45480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.303181:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346286720 00000100:00100000:0.0:1787826345.303185:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346286720 00000100:00100000:0.0:1787826345.303205:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23942 00000100:00100000:0.0:1787826345.303211:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286720:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.303263:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286720:12345-0@lo:1000:mdt_io00_002.0 Request processed in 51us (106us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.303269:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23942 00000100:00100000:0.0:1787826345.303295:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286720:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826345.303310:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e56:0x0] 00000100:00100000:0.0:1787826345.303333:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286848:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.303356:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.303370:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d45480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.303381:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346286848 00000100:00100000:0.0:1787826345.303384:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346286848 00000100:00100000:0.0:1787826345.303404:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23943 00000100:00100000:0.0:1787826345.303421:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286848:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.303480:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286848:12345-0@lo:1000:mdt_io00_002.0 Request processed in 69us (122us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.303486:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23943 00000100:00100000:0.0:1787826345.303512:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286848:0@lo:1000:mdt_io00_002.0 00000004:00080000:0.0:1787826345.303539:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x199:0x0] 00000100:00100000:0.0:1787826345.303557:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286976:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.303580:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.303592:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d45480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.303603:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346286976 00000100:00100000:0.0:1787826345.303606:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346286976 00000100:00100000:0.0:1787826345.303625:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23944 00000100:00100000:0.0:1787826345.303630:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286976:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.303673:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346286976:12345-0@lo:1000:mdt_io00_002.0 Request processed in 43us (92us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.303679:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23944 00000100:00100000:0.0:1787826345.303703:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346286976:0@lo:1000:mdt_io00_002.0 00000004:00020000:0.0:1787826345.303725:0:6542:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:0.0:1787826345.303772:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.303781:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.303789:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.303801:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346287104:0@lo:101:mdt_io00_002.0 00000100:00100000:0.0:1787826345.303823:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.303835:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d45d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.303845:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346287104 00000100:00100000:0.0:1787826345.303848:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346287104 00000100:00100000:0.0:1787826345.303868:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23945 00000100:00100000:0.0:1787826345.303873:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6542:x1874663346287104:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826345.303881:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826345.303893:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4baca lrc: 2/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bac3 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.303914:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.303922:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4baca lrc: 4/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bac3 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.303951:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.303964:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 6 pid: 35094 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.303975:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346287232:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826345.304021:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d45480 going to sleep for 11 seconds 00000004:00020000:3.0:1787826345.304875:0:35093:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4476 00010000:00010000:3.0:1787826345.304957:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.304969:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.304980:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826345.304990:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dbf:0x0]: returning size 6 00010000:00010000:3.0:1787826345.305084:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305091:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305097:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305137:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305145:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826345.305150:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305157:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305169:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8b00) 00010000:00010000:3.0:1787826345.305172:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826345.305176:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305182:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ba45 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305922:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305942:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305949:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305955:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826345.305959:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305966:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305976:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f79460a2900) 00010000:00010000:3.0:1787826345.305979:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826345.305982:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.305988:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ba4c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.306062:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ba3e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.306083:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.306090:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f7843326100) 00010000:00010000:3.0:1787826345.306092:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826345.306094:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826345.306183:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225995:x1874663332648064:12345-192.168.203.56@tcp:101:cat.0 Request processed in 527197us (534892us total) trans 8589957449 rc 0/0 00000100:00100000:3.0:1787826345.306190:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68580 00000100:00100000:0.0:1787826345.311361:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346287232:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826345.312839:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826345.312892:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332650368 00000100:00100000:3.0:1787826345.312901:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332650368 00010000:00010000:3.0:1787826345.312956:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 7 pid: 35094 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.313001:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22017 00000100:00100000:3.0:1787826345.313013:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332650368:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826345.313028:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826345.313036:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 7 pid: 35094 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313049:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 7 pid: 35094 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313072:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef800/0xe21309aa81f4ba29 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018af expref: 7 pid: 35094 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313091:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:3.0:1787826345.313098:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4baca lrc: 3/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bac3 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313131:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4baca lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4bac3 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313140:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4baca lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4bac3 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313172:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4baca lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bac3 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.313188:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001_UUID:125406:1874663346287360:0@lo:105: 00000100:00100000:3.0:1787826345.313234:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.313270:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826345.313287:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346287360 00000100:00100000:3.0:1787826345.313291:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346287360 00000100:00100000:3.0:1787826345.313318:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7900 00000100:00100000:3.0:1787826345.313324:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:125406:x1874663346287360:12345-0@lo:105: 00010000:00010000:3.0:1787826345.313335:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313345:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313359:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313367:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313389:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.313391:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4baca lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bac3 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.313402:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.313406:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:0.0:1787826345.313421:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826345.313425:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00010000:00010000:3.0:1787826345.313431:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43a00) 00000100:00100000:3.0:1787826345.313465:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:125406:x1874663346287360:12345-0@lo:105: Request processed in 140us (230us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.313472:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7900 00000100:00100000:3.0:1787826345.313509:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001_UUID:125406:1874663346287360:0@lo:105: 00010000:00010000:3.0:1787826345.313522:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826345.313556:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332650368:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 547us (726us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.313563:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22017 00000100:00100000:0.0:1787826345.313802:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346287104:12345-0@lo:101:mdt_io00_002.0 Request processed in 9927us (9976us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.313810:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23945 00000100:00100000:1.0:1787826345.316264:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346287104:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826345.316292:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826345.316318:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x199:0x0] 00010000:00010000:1.0:1787826345.316365:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.316383:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.316392:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826345.316421:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346287488:0@lo:101:mdt_io00_002.0 00000100:00100000:1.0:1787826345.316482:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.316522:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826345.316570:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826345.316596:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332650496 00000100:00100000:1.0:1787826345.316602:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332650496 00000100:00100000:1.0:1787826345.316649:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30863 00000100:00100000:3.0:1787826345.316658:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346287488 00000100:00100000:1.0:1787826345.316660:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332650496:12345-192.168.203.56@tcp:101:cat.0 00000100:00100000:3.0:1787826345.316665:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346287488 00010000:00010000:1.0:1787826345.316676:0:35097:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826345.316690:0:35097:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009932c09f/0xe21309aa81f4bae6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018e7 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826345.316729:0:35097:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:35097:1874663346287616:192.168.203.56@tcp:106: 00000100:00100000:3.0:1787826345.316875:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23946 00000100:00100000:3.0:1787826345.316884:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346287488:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:3.0:1787826345.316894:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826345.316910:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4baed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4badf expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.316944:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4baed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4badf expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.316959:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4baed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4badf expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.316968:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x4ec94fd6 (ffff9f7843326f00) 00010000:00010000:3.0:1787826345.316971:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826345.316974:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826345.317014:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346287488:12345-0@lo:101:mdt_io00_002.0 Request processed in 132us (532us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.317021:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23946 00000100:00100000:1.0:1787826345.318515:0:35097:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826345.318560:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346287488:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826345.318580:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4baed expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.318592:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4baed expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826345.318616:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x199:0x0] 00010000:00010000:1.0:1787826345.318640:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.318653:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.318661:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.318689:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.318703:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 10 pid: 8752 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826345.318714:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346287744:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826345.320164:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00002000:00010000:3.0:1787826345.323503:0:35097:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x2c0000401:0x522:0x0] updating lvb size: 0 -> 6 00002000:00010000:3.0:1787826345.323512:0:35097:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x2c0000401:0x522:0x0] updating lvb mtime: 0 -> 1787826340 00002000:00010000:3.0:1787826345.323516:0:35097:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x2c0000401:0x522:0x0] updating lvb atime: 0 -> 1787826340 00002000:00010000:3.0:1787826345.323518:0:35097:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x2c0000401:0x522:0x0] updating lvb ctime: 0 -> 1787826340 00002000:00010000:3.0:1787826345.323521:0:35097:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x2c0000401:0x522:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:3.0:1787826345.323548:0:35097:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0001_UUID:35097:1874663346287616:192.168.203.56@tcp:106: 00010000:00010000:3.0:1787826345.323590:0:35097:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009932c09f/0xe21309aa81f4bae6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018e7 expref: 6 pid: 35097 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.323614:0:35097:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009932c09f/0xe21309aa81f4bae6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018e7 expref: 6 pid: 35097 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.323631:0:35097:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:3.0:1787826345.323778:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332650496:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7118us (7207us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826345.323788:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30863 00000100:00100000:1.0:1787826345.324178:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.324275:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332650624 00000100:00100000:0.0:1787826345.324284:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332650624 00010000:00010000:0.0:1787826345.324321:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 11 pid: 8752 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.324340:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346287744:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826345.324362:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22018 00010000:00010000:3.0:1787826345.324362:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.324385:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332650624:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826345.324404:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826345.324421:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 11 pid: 8752 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.324435:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 11 pid: 8752 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.324453:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4ba37 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018b6 expref: 11 pid: 8752 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.324466:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8b00) 00010000:00010000:0.0:1787826345.324471:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.324481:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.324489:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.324514:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826345.324541:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332650624:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 160us (364us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.324549:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22018 00010000:00010000:3.0:1787826345.326494:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.326509:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.326536:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.326548:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.326558:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.326631:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346287872:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.326695:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.326739:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.326778:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346287872 00000100:00100000:0.0:1787826345.326787:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346287872 00000100:00100000:0.0:1787826345.326841:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23947 00000100:00100000:0.0:1787826345.326850:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346287872:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.326940:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346287872:12345-0@lo:1000:mdt_io00_002.0 Request processed in 84us (237us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.326947:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23947 00000100:00100000:3.0:1787826345.327285:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346287872:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.327634:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288000:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.327669:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.327788:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.327823:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346288000 00000100:00100000:0.0:1787826345.327829:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346288000 00000100:00100000:0.0:1787826345.327893:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23948 00000100:00100000:0.0:1787826345.327901:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288000:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.327996:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288000:12345-0@lo:1000:mdt_io00_002.0 Request processed in 96us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.328003:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23948 00000100:00100000:1.0:1787826345.328078:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288000:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.328145:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288128:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.328179:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.328207:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826345.328273:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346288128 00000100:00100000:3.0:1787826345.328278:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346288128 00000100:00100000:3.0:1787826345.328313:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23949 00000100:00100000:3.0:1787826345.328322:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288128:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.328445:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288128:12345-0@lo:1000:mdt_io00_002.0 Request processed in 124us (267us total) trans 0 rc 1/1 00000100:00100000:3.0:1787826345.328451:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23949 00000100:00100000:1.0:1787826345.329393:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288128:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.329495:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288256:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.329538:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.329580:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826345.330020:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346288256 00000100:00100000:3.0:1787826345.330038:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346288256 00000100:00100000:3.0:1787826345.330145:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23950 00000100:00100000:3.0:1787826345.330153:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288256:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.330295:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288256:12345-0@lo:1000:mdt_io00_002.0 Request processed in 141us (758us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.330301:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23950 00000100:00100000:1.0:1787826345.331050:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288256:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.331166:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288384:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.331201:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.331227:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826345.331313:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346288384 00000100:00100000:3.0:1787826345.331319:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346288384 00000100:00100000:3.0:1787826345.331587:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23951 00000100:00100000:3.0:1787826345.331597:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288384:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.331704:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288384:12345-0@lo:1000:mdt_io00_002.0 Request processed in 108us (502us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.331712:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23951 00000100:00100000:2.0:1787826345.331851:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288384:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787826345.331879:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e56:0x0] 00010000:00010000:2.0:1787826345.332237:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.332255:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.332263:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.332297:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.332311:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 9 pid: 35093 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826345.332323:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346288512:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826345.332387:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826345.347036:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826345.347328:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332650752 00000100:00100000:3.0:1787826345.347343:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332650752 00000100:00100000:3.0:1787826345.347443:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30864 00000100:00100000:3.0:1787826345.347454:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:225995:x1874663332650752:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826345.347472:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826345.347492:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018ee expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.347527:0:24218:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.347536:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x522:0x0].0x0 since=4s v=(0 0) 00010000:00010000:3.0:1787826345.347541:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826345.347546:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.347582:0:24218:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.347595:0:24218:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 5 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.347610:0:24218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:24218:1874663346288640:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826345.347702:0:24218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826345.349819:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346288512:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826345.349853:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.353730:0:24218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:24218:1874663346288640:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826345.353776:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.353797:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:0.0:1787826345.353864:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:225995:x1874663332650752:12345-192.168.203.56@tcp:101:cat.0 Request processed in 6411us (6838us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.353873:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30864 00000100:00100000:1.0:1787826345.357457:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826345.357509:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826345.357608:0:20151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332651008 00000100:00100000:3.0:1787826345.357618:0:20151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332651008 00002000:00100000:3.0:1787826345.357792:0:20151:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x522:0x0] (0->5) req@ffff9f785124f480 x1874663332651008/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:701/0 lens 496/0 e 0 to 0 dl 1787826356 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826345.357810:0:20151:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124f480 with x1874663332651008 ext(0->5) 00010000:00010000:3.0:1787826345.357822:0:20151:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 6 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.357836:0:20151:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 6 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826345.357844:0:20151:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9f785124f480 00000100:00100000:3.0:1787826345.357881:0:20151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2491 00000100:00100000:3.0:1787826345.357892:0:20151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332651008:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:0.0:1787826345.358386:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332650880 00000100:00100000:0.0:1787826345.358392:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332650880 00010000:00010000:0.0:1787826345.358427:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 10 pid: 35093 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.358460:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22019 00000100:00100000:0.0:1787826345.358468:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332650880:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826345.358482:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826345.358489:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 10 pid: 35093 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.358502:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 10 pid: 35093 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.358523:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bad1 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018bd expref: 10 pid: 35093 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.358540:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f7843326100) 00010000:00010000:0.0:1787826345.358546:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.358555:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.358562:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.358590:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826345.358605:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.358616:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.358644:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332650880:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 176us (1197us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.358651:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22019 00010000:00010000:3.0:1787826345.358904:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.358913:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.358923:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.358947:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288768:0@lo:101:mdt_io00_002.0 00000100:00100000:3.0:1787826345.358997:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826345.359757:0:20151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332651008:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1861us (2243us total) trans 4294976166 rc 0/0 00000100:00100000:2.0:1787826345.359768:0:20151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2491 00002000:00100000:2.0:1787826345.359779:0:20151:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x522:0x0] (0->5) req@ffff9f785124f480 x1874663332651008/t4294976166(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:701/0 lens 496/448 e 0 to 0 dl 1787826356 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826345.359789:0:20151:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124f480 with x1874663332651008 ext(0->5) 00010000:00010000:2.0:1787826345.359808:0:20151:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 8 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.359825:0:20151:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 8 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826345.359834:0:20151:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9f785124f480 00000100:00100000:2.0:1787826345.360366:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.367260:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346288768 00000100:00100000:0.0:1787826345.367285:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346288768 00000100:00100000:0.0:1787826345.367378:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23952 00000100:00100000:0.0:1787826345.367389:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346288768:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826345.367422:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826345.367450:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4bb17 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bb10 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.367481:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4bb17 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bb10 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.367495:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4bb17 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bb10 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.367503:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7845784c00) 00010000:00010000:0.0:1787826345.367505:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787826345.367509:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826345.367607:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346288768:12345-0@lo:101:mdt_io00_002.0 Request processed in 219us (8609us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.367613:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23952 00000100:00100000:2.0:1787826345.368038:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288768:0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787826345.368061:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bb17 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.368074:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bb17 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826345.368102:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e57:0x0] 00000100:00100000:2.0:1787826345.368196:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288896:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826345.368301:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.368348:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826345.368372:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.368710:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332651136 00000100:00100000:0.0:1787826345.368716:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332651136 00010000:00010000:0.0:1787826345.368748:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 8 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.368777:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22020 00000100:00100000:0.0:1787826345.368787:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332651136:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826345.368798:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1787826345.368815:0:125406:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000401:0x522:0x0] updating lvb blocks from disk: 1 -> 8 00010000:00010000:0.0:1787826345.368822:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 8 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.368839:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 8 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.369041:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4b9b2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801885 expref: 8 pid: 35352 timeout: 8257 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.369058:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826345.369063:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826345.369071:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.369081:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.369464:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.369480:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:125406:1874663346289024:192.168.203.56@tcp:105: 00000100:00100000:2.0:1787826345.369852:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d45480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826345.372394:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346288896 00000100:00100000:3.0:1787826345.372408:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346288896 00000100:00100000:3.0:1787826345.373818:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851837b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:125406:1874663346289024:192.168.203.56@tcp:105: 00010000:00010000:3.0:1787826345.373848:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826345.373908:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2396:x1874663332651136:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 5121us (5561us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.373919:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22020 00000100:00100000:0.0:1787826345.376821:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826345.376913:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332651264 00000100:00100000:1.0:1787826345.376923:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332651264 00000100:00100000:1.0:1787826345.376996:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68586 00000100:00100000:1.0:1787826345.377009:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:225995:x1874663332651264:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826345.377036:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826345.377059:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bb1e lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018f5 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.377138:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bb1e lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018f5 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.377152:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bb1e lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018f5 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826345.377171:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f7843326100) 00010000:00010000:1.0:1787826345.377175:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787826345.377178:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826345.377248:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225995:x1874663332651264:12345-192.168.203.56@tcp:101:cat.0 Request processed in 241us (434us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.377254:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68586 00000100:00100000:1.0:1787826345.382724:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23953 00000100:00100000:1.0:1787826345.382747:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346288896:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.387347:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346288896:12345-0@lo:1000:mdt_io00_002.0 Request processed in 4600us (19087us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.387364:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23953 00000100:00100000:3.0:1787826345.387586:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346288896:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826345.387712:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x199:0x0] 00000100:00100000:3.0:1787826345.387772:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289152:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.389066:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.389135:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.389445:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346289152 00000100:00100000:0.0:1787826345.389453:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346289152 00000100:00100000:0.0:1787826345.389507:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23954 00000100:00100000:0.0:1787826345.389518:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289152:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.389614:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289152:12345-0@lo:1000:mdt_io00_002.0 Request processed in 98us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.389623:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23954 00000100:00100000:1.0:1787826345.390006:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.390361:0:20179:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332651392 00000100:00100000:0.0:1787826345.390366:0:20179:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332651392 00002000:00100000:0.0:1787826345.390404:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x522:0x0] (0->4095) req@ffff9f785124d880 x1874663332651392/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:701/0 lens 488/0 e 0 to 0 dl 1787826356 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826345.390427:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124d880 with x1874663332651392 ext(0->4095) 00002000:00010000:0.0:1787826345.390433:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f785124d880 00000100:00100000:0.0:1787826345.390463:0:20179:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27659 00000100:00100000:0.0:1787826345.390472:0:20179:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332651392:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:0.0:1787826345.390675:0:20179:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332651392:12345-192.168.203.56@tcp:3:cat.0 Request processed in 203us (671us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1787826345.390683:0:20179:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27659 00002000:00100000:0.0:1787826345.390691:0:20179:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x522:0x0] (0->4095) req@ffff9f785124d880 x1874663332651392/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:701/0 lens 488/4536 e 0 to 0 dl 1787826356 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826345.390699:0:20179:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124d880 with x1874663332651392 ext(0->4095) 00002000:00010000:0.0:1787826345.390704:0:20179:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f785124d880 00000100:00100000:2.0:1787826345.391282:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289152:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787826345.391329:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e56:0x0] 00010000:00010000:2.0:1787826345.391372:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.391387:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.391397:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826345.391486:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289280:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826345.391535:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826345.393890:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.398089:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346289280 00000100:00100000:0.0:1787826345.398548:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346289280 00000100:00100000:1.0:1787826345.399008:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23955 00000100:00100000:1.0:1787826345.399021:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289280:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.399521:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289280:12345-0@lo:1000:mdt_io00_002.0 Request processed in 499us (7982us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.399532:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23955 00000100:00100000:2.0:1787826345.399563:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289280:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787826345.399616:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e56:0x0] 00000100:00100000:2.0:1787826345.399672:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289408:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826345.399712:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826345.399747:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.399806:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346289408 00000100:00100000:0.0:1787826345.399812:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346289408 00000100:00100000:0.0:1787826345.399907:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23956 00000100:00100000:0.0:1787826345.399917:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289408:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.399988:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289408:12345-0@lo:1000:mdt_io00_002.0 Request processed in 73us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.399995:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23956 00000100:00100000:2.0:1787826345.400057:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289408:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826345.400095:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289536:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826345.400138:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.401050:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346289536 00000100:00100000:0.0:1787826345.401059:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346289536 00000100:00100000:2.0:1787826345.401082:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.401101:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23957 00000100:00100000:0.0:1787826345.401137:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289536:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.401237:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289536:12345-0@lo:1000:mdt_io00_002.0 Request processed in 102us (1097us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.401246:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23957 00000100:00100000:2.0:1787826345.401346:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289536:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787826345.401539:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x199:0x0] 00000100:00100000:2.0:1787826345.401585:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289664:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826345.401617:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826345.401641:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826345.401779:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346289664 00000100:00100000:1.0:1787826345.401785:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346289664 00000100:00100000:1.0:1787826345.401837:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23958 00000100:00100000:1.0:1787826345.401855:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289664:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.402147:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289664:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826345.402172:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e56:0x0] 00000100:00100000:3.0:1787826345.402208:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289792:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.402225:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289664:12345-0@lo:1000:mdt_io00_002.0 Request processed in 79us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.402233:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23958 00000100:00100000:3.0:1787826345.402242:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.402254:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346289792 00000100:00100000:1.0:1787826345.402259:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346289792 00000100:00100000:3.0:1787826345.402261:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826345.402287:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23959 00000100:00100000:1.0:1787826345.402295:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289792:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826345.402353:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289792:12345-0@lo:1000:mdt_io00_002.0 Request processed in 58us (112us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.402359:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23959 00000100:00100000:3.0:1787826345.402570:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289792:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.402606:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289920:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.402634:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.402658:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826345.402905:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346289920 00000100:00100000:0.0:1787826345.402911:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346289920 00000100:00100000:0.0:1787826345.402952:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23960 00000100:00100000:0.0:1787826345.402961:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf88450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289920:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826345.403074:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf88450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346289920:12345-0@lo:1000:mdt_io00_002.0 Request processed in 114us (438us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.403081:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23960 00000100:00100000:3.0:1787826345.403492:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346289920:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.403574:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346290048:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826345.403651:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.403680:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826345.403903:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346290048 00000100:00100000:2.0:1787826345.403909:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346290048 00000100:00100000:2.0:1787826345.404068:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23961 00000100:00100000:2.0:1787826345.404078:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346290048:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826345.404198:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346290048:12345-0@lo:1000:mdt_io00_002.0 Request processed in 122us (546us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.404207:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23961 00000100:00100000:3.0:1787826345.404236:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346290048:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826345.404669:0:6542:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1dbf:0x0] is inserted into orphan list, open count = 1 00000040:00080000:3.0:1787826345.404842:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].941, 1240 off885976 00000040:00080000:3.0:1787826345.405035:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].944, 1320 off985680 00000004:00080000:3.0:1787826345.405042:0:6542:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f78434e4d80 with version 1834 00000004:00080000:0.0:1787826345.405404:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1834 rpc_version 1834 00000020:00100000:0.0:1787826345.405442:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826345.405449:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7851834000 x1874663346290304/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826345.405467:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851834000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346290304:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826345.405512:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.411777:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346290304 00000100:00100000:1.0:1787826345.411792:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346290304 00000100:00100000:0.0:1787826345.411970:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7845d459c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826345.416043:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23962 00000100:00100000:1.0:1787826345.416071:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346290304:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826345.416196:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826345.416229:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332651648 00000100:00100000:0.0:1787826345.416237:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332651648 00000100:00100000:0.0:1787826345.416279:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9375 00000100:00100000:0.0:1787826345.416286:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:225995:x1874663332651648:12345-192.168.203.56@tcp:35:cat.0 00000100:00100000:1.0:1787826345.416464:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346290304:12345-0@lo:1000:osp_up1-0.0 Request processed in 406us (10949us total) trans 4294985449 rc 0/0 00000004:00080000:0.0:1787826345.416466:0:6530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1dbf:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1787826345.416523:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].215, 64 off21952 00000004:00080000:0.0:1787826345.416534:0:6530:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8157628427929ns, next at 8158628425055ns, rc = 0 00010000:00010000:0.0:1787826345.416581:0:6530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.416596:0:6530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.416607:0:6530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.416629:0:6530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.416636:0:6530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.416641:0:6530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.416657:0:6530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826345.418906:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:225995:x1874663332651648:12345-192.168.203.56@tcp:35:cat.0 Request processed in 2609us (2702us total) trans 8589957451 rc 0/0 00000100:00100000:0.0:1787826345.418919:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9375 00010000:00010000:0.0:1787826345.418979:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826345.419100:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.419133:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826345.419137:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.419149:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.419165:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f7843326100) 00010000:00010000:0.0:1787826345.419168:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826345.419173:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.419179:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bb2c lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dbf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.419193:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826345.419194:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826345.419196:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826345.419256:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851834000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346290304:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787826345.420326:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420338:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420345:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420351:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826345.420355:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420363:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420375:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f7843326100) 00010000:00010000:3.0:1787826345.420378:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826345.420382:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420388:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4bb02 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826345.420405:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e57:0x0] 00010000:00010000:3.0:1787826345.420425:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bb17 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420435:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bb17 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420452:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420459:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420466:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420471:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826345.420474:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420480:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420488:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8b00) 00010000:00010000:3.0:1787826345.420491:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826345.420494:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420501:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4baf4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826345.420520:0:6542:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b277643d reply state 000000008de41782 transno 8589957450 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420537:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420544:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f79460a2900) 00010000:00010000:3.0:1787826345.420546:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826345.420549:0:6542:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:3.0:1787826345.420552:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x199:0x0] 00010000:00010000:3.0:1787826345.420557:0:6542:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420564:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420571:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420594:0:6542:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826345.420599:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x199:0x0] 00010000:00010000:3.0:1787826345.420605:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4baed expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420612:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4baed expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420632:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420638:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420645:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420650:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826345.420653:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420659:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420667:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7967ac8000) 00010000:00010000:3.0:1787826345.420669:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826345.420672:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.420787:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4bb25 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.421013:0:6542:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:225997:x1874663332650240:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 119025us (119191us total) trans 8589957450 rc 0/0 00000100:00100000:3.0:1787826345.421024:0:6542:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1234 00010000:00010000:0.0:1787826345.421224:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826345.421231:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.421240:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826345.421245:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826345.421254:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.421289:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.421302:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826345.421327:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.421335:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43a00/0xe21309aa81f4bac3 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4baca expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.421351:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826345.421352:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826345.421354:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826345.421457:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826345.421477:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851837100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346290432:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826345.421534:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.421628:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346290432 00000100:00100000:0.0:1787826345.421635:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346290432 00000100:00100000:0.0:1787826345.421694:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11166 00000100:00100000:0.0:1787826345.421704:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b471050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3655:x1874663346290432:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826345.421716:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826345.421733:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4baca lrc: 0/0,0 mode: --/CW res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bac3 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.421746:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f7967ac8900) 00010000:00010000:0.0:1787826345.421755:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826345.422048:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b471050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346290432:12345-0@lo:103:ldlm_bl.0 Request processed in 345us (513us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.422056:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11166 00000100:00100000:0.0:1787826345.422154:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851837100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346290432:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826345.431449:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23962 00000100:00100000:1.0:1787826345.965956:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346290688:0@lo:400:kworker.0 00000100:00100000:1.0:1787826345.966034:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.966145:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826345.966151:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346290816:0@lo:400:kworker.0 00000100:00100000:1.0:1787826345.966179:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.966225:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346291200:0@lo:400:kworker.0 00000100:00100000:1.0:1787826345.966248:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.966292:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346291200 00000100:00100000:1.0:1787826345.966300:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346291200 00000100:00100000:1.0:1787826345.966362:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8859 00000100:00100000:1.0:1787826345.966371:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346291200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826345.966445:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346291200:12345-0@lo:400:kworker.0 Request processed in 75us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826345.966453:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8859 00000100:00080000:1.0:1787826345.966475:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826345.966504:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973855 gen 2 00000004:00080000:1.0:1787826345.966509:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784f541500, transno 4294973855 00000100:00100000:1.0:1787826345.966521:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346291200:0@lo:400:kworker.0 00000100:00100000:1.0:1787826345.966557:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826345.966563:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346291328:0@lo:400:kworker.0 00000100:00100000:1.0:1787826345.966586:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.966628:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346291712:0@lo:400:kworker.0 00000100:00100000:1.0:1787826345.966650:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.966783:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346290560:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.966832:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.966864:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346291072:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.966888:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.966906:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346291584:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.966929:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.966950:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1787826345.966955:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346290944:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.966977:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.966994:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346291456:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967022:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.967061:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346290816 00000100:00100000:0.0:1787826345.967065:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346290816 00000100:00100000:0.0:1787826345.967123:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346291328 00000100:00100000:0.0:1787826345.967127:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346291328 00000100:00100000:3.0:1787826345.967139:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346290688 00000100:00100000:3.0:1787826345.967148:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346290688 00000100:00100000:0.0:1787826345.967159:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346291072 00000100:00100000:0.0:1787826345.967164:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346291072 00000100:00100000:0.0:1787826345.967191:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346290944 00000100:00100000:0.0:1787826345.967194:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346290944 00000100:00100000:0.0:1787826345.967221:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346291456 00000100:00100000:0.0:1787826345.967224:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346291456 00000100:00100000:0.0:1787826345.967240:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14348 00000100:00100000:0.0:1787826345.967247:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346290816:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967287:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346290816:12345-0@lo:400:kworker.0 Request processed in 40us (1107us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.967293:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14348 00000100:00100000:0.0:1787826345.967308:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14349 00000100:00100000:0.0:1787826345.967315:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346291328:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967341:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346291328:12345-0@lo:400:kworker.0 Request processed in 27us (754us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.967350:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14349 00000100:00100000:0.0:1787826345.967361:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14350 00000100:00100000:0.0:1787826345.967367:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346291072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967396:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346291072:12345-0@lo:400:kworker.0 Request processed in 30us (507us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.967402:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14350 00000100:00100000:0.0:1787826345.967428:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14351 00000100:00100000:0.0:1787826345.967435:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346290944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967463:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346290944:12345-0@lo:400:kworker.0 Request processed in 29us (485us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.967469:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14351 00000100:00100000:0.0:1787826345.967480:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14352 00000100:00100000:0.0:1787826345.967486:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346291456:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967512:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346291456:12345-0@lo:400:kworker.0 Request processed in 27us (489us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.967519:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14352 00000100:00100000:0.0:1787826345.967654:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346290560 00000100:00100000:0.0:1787826345.967658:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346290560 00000100:00100000:0.0:1787826345.967700:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2235 00000100:00100000:0.0:1787826345.967707:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79747d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346290560:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967737:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79747d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346290560:12345-0@lo:400:kworker.0 Request processed in 31us (905us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.967743:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2235 00000100:00100000:0.0:1787826345.967776:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346291584 00000100:00100000:0.0:1787826345.967780:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346291584 00000100:00100000:0.0:1787826345.967809:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8860 00000100:00100000:0.0:1787826345.967814:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3655:x1874663346291584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967844:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3655:x1874663346291584:12345-0@lo:400:kworker.0 Request processed in 30us (914us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.967849:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8860 00000100:00080000:0.0:1787826345.967885:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787826345.967888:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787826345.967890:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826345.967913:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346290944:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967922:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346291456:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967931:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346290560:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967940:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346291072:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.967949:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346291584:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.968000:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346291328:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.968011:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346290816:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.968074:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346291840:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.968100:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.968189:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346291968:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.968213:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826345.968493:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346291712 00000100:00100000:1.0:1787826345.968498:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346291712 00000100:00100000:3.0:1787826345.969550:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23963 00000100:00100000:3.0:1787826345.969566:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346290688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826345.969646:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346290688:12345-0@lo:400:kworker.0 Request processed in 86us (3611us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.969654:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23963 00000100:00100000:3.0:1787826345.969672:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23964 00000100:00100000:3.0:1787826345.969679:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346291712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826345.969710:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:3652:x1874663346291712:12345-0@lo:400:kworker.0 Request processed in 32us (3059us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.969716:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23964 00000100:00100000:0.0:1787826345.969989:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346291712:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.970004:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346290688:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.974333:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346292096:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.974421:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826345.974474:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346292096 00000100:00100000:0.0:1787826345.974481:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346292096 00000100:00100000:0.0:1787826345.974537:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14353 00000100:00100000:0.0:1787826345.974546:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346292096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.974596:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346292096:12345-0@lo:400:kworker.0 Request processed in 51us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.974604:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14353 00000100:00100000:0.0:1787826345.974646:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346292096:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.974857:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346291840 00000100:00100000:0.0:1787826345.974862:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346291840 00000100:00100000:0.0:1787826345.974906:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346291968 00000100:00100000:0.0:1787826345.974910:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346291968 00000100:00100000:0.0:1787826345.974934:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8861 00000100:00100000:0.0:1787826345.974941:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346291840:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.974985:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346291840:12345-0@lo:400:kworker.0 Request processed in 44us (6883us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.974991:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8861 00000100:00100000:0.0:1787826345.975004:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8862 00000100:00100000:0.0:1787826345.975011:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346291968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.975044:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346291968:12345-0@lo:400:kworker.0 Request processed in 34us (6830us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826345.975051:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8862 00000100:00080000:0.0:1787826345.975065:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787826345.975069:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826345.975086:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973856 gen 2 00000004:00080000:0.0:1787826345.975093:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7850ad4e00, transno 4294973856 00000100:00100000:0.0:1787826345.975131:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847707b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346291840:0@lo:400:kworker.0 00000100:00100000:0.0:1787826345.975332:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346291968:0@lo:400:kworker.0 00010000:00010000:3.0:1787826345.982679:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4baed expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826345.982700:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4baed expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.982749:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4baed expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.982756:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826345.982779:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4badf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4baed expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.982801:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bb17 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826345.982810:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bb17 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.982827:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bb17 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.982838:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826345.982844:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4bb10 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bb17 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826345.982857:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957451 is committed 00000004:00080000:3.0:1787826345.982866:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 220 committed 00000001:00080000:3.0:1787826345.982897:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957450 is committed 00000020:00080000:3.0:1787826345.982910:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979086 batchid updated 00000001:00080000:3.0:1787826345.982920:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957448 is committed 00010000:00010000:3.0:1787826345.983018:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983027:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983035:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983041:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826345.983047:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983055:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983069:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f79460a2900) 00010000:00010000:3.0:1787826345.983072:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826345.983077:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983087:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bafb lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787826345.983137:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979086 00000004:00080000:3.0:1787826345.983196:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f78434e4000 with version 1835 00000020:00080000:3.0:1787826345.983207:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979086 cancel update log [0x240000404:0x2:0x0].943: rc = 0 00000020:00080000:3.0:1787826345.983279:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979086 cancel update log [0x200000bd0:0x2:0x0].939: rc = 0 00000020:00080000:3.0:1787826345.983287:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979086 00000004:00080000:3.0:1787826345.983295:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1835 rpc_version 1835 00000020:00100000:3.0:1787826345.983313:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826345.983324:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9eaa00 x1874663346292736/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826345.983476:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346292736:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826345.983545:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.983576:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3b0f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826345.983592:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346292352:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826345.983620:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.983641:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826345.983647:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346292224:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826345.983666:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.983740:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346292352 00000100:00100000:3.0:1787826345.983747:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346292352 00000100:00100000:3.0:1787826345.983788:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346292224 00000100:00100000:2.0:1787826345.983791:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346292736 00000100:00100000:3.0:1787826345.983792:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346292224 00000100:00100000:2.0:1787826345.983795:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346292736 00000100:00100000:3.0:1787826345.983813:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11167 00000100:00100000:2.0:1787826345.983819:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23965 00000100:00100000:3.0:1787826345.983821:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3653:x1874663346292352:12345-0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826345.983824:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:30909:x1874663346292736:12345-0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787826345.983833:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826345.983854:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4bb17 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bb10 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983868:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7845784c00) 00010000:00010000:3.0:1787826345.983875:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826345.983907:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3653:x1874663346292352:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 87us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826345.983913:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11167 00000100:00100000:3.0:1787826345.983931:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11168 00000100:00100000:3.0:1787826345.983937:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346292224:12345-0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826345.983939:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346292736:12345-0@lo:1000:osp_up1-0.0 Request processed in 115us (394us total) trans 4294985450 rc 0/0 00010000:00010000:3.0:1787826345.983944:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787826345.983946:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23965 00010000:00010000:3.0:1787826345.983954:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4baed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x199:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4badf expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983963:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x4ec94fd6 (ffff9f7843326f00) 00010000:00010000:2.0:1787826345.983965:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.983970:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826345.983972:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.983978:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.983984:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826345.983987:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.983989:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346292224:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 53us (322us total) trans 0 rc 0/0 00010000:00010000:2.0:1787826345.983994:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.983995:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11168 00010000:00010000:2.0:1787826345.984003:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f79460a2900) 00010000:00010000:2.0:1787826345.984006:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826345.984009:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.984014:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4b99d lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.984037:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eaa00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346292736:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826345.984052:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346292480:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1787826345.984056:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346292224:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826345.984063:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346292352:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826345.984084:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826345.984100:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346292480 00000100:00100000:3.0:1787826345.984119:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346292480 00000100:00100000:3.0:1787826345.984159:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30865 00000100:00100000:3.0:1787826345.984167:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346292480:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1787826345.984184:0:20167:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1314 count 1 00010000:00010000:3.0:1787826345.984206:0:20167:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.984225:0:20167:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.984233:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=1s v=(1 0) 00010000:00010000:3.0:1787826345.984237:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826345.984241:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.984264:0:20167:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826345.984278:0:20167:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 5 pid: 24218 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826345.984289:0:20167:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20167:1874663346292864:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826345.985252:0:20167:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3b0f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826345.997239:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826345.997327:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332651776 00000100:00100000:2.0:1787826345.997350:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332651776 00010000:00010000:2.0:1787826345.997394:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 6 pid: 24218 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826345.997448:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22021 00000100:00100000:2.0:1787826345.997459:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332651776:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826345.997474:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826345.997480:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 6 pid: 24218 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.997494:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 6 pid: 24218 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.997521:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826345.997525:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826345.997531:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.997542:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826345.997563:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826345.997617:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332651776:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 159us (389us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826345.997625:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22021 00010000:00010000:0.0:1787826345.998236:0:20167:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4bb09 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608018ee expref: 5 pid: 24218 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826345.998254:0:20167:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20167:1874663346292864:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826345.998290:0:20167:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.998301:0:20167:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.998309:0:20167:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.998319:0:20167:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.998327:0:20167:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826345.998350:0:20167:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826345.998549:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346292480:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14381us (14462us total) trans 4294976167 rc 0/0 00000100:00100000:0.0:1787826345.998558:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30865 00010000:00010000:1.0:1787826346.001478:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826346.001495:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.001509:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826346.001513:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.001524:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.001544:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.001558:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f94a9c48/0xe21309aa81f4bb33 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.001583:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826346.001585:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826346.001587:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1787826346.001703:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784d9e9500/2, rc 0, transno 4294976167 00000100:00100000:2.0:1787826346.001713:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346292480:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1787826346.159724:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826346.209210:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826346.209294:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.210165:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332652032 00000100:00100000:3.0:1787826346.210176:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332652032 00000100:00100000:3.0:1787826346.210347:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30866 00000100:00100000:3.0:1787826346.210358:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332652032:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:3.0:1787826346.210486:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332652032:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 130us (1191us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.210493:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30866 00000100:00100000:2.0:1787826346.216300:0:8449:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332651904 00000100:00100000:2.0:1787826346.216315:0:8449:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332651904 00002000:00100000:2.0:1787826346.216370:0:8449:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4cf:0x0] (0->5) req@ffff9f785124d500 x1874663332651904/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:702/0 lens 496/0 e 0 to 0 dl 1787826357 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826346.216383:0:8449:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124d500 with x1874663332651904 ext(0->5) 00002000:00010000:2.0:1787826346.216389:0:8449:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f785124d500 00000100:00100000:2.0:1787826346.216466:0:8449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27660 00000100:00100000:2.0:1787826346.216478:0:8449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332651904:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826346.218062:0:8449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332651904:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1578us (8854us total) trans 4294973857 rc 0/0 00000100:00100000:2.0:1787826346.218074:0:8449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27660 00002000:00100000:2.0:1787826346.218085:0:8449:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4cf:0x0] (0->5) req@ffff9f785124d500 x1874663332651904/t4294973857(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:702/0 lens 496/448 e 0 to 0 dl 1787826357 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826346.218095:0:8449:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785124d500 with x1874663332651904 ext(0->5) 00002000:00010000:2.0:1787826346.218101:0:8449:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f785124d500 00000100:00100000:0.0:1787826346.482660:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826346.482700:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346292992:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826346.482771:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826346.482824:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346292992 00000100:00100000:0.0:1787826346.482831:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346292992 00000100:00100000:0.0:1787826346.482887:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6251 00000100:00100000:0.0:1787826346.482897:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346292992:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826346.482918:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826346.482963:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346292992:12345-0@lo:13:osp-pre-1-1.0 Request processed in 66us (191us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826346.482970:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6251 00000100:00100000:0.0:1787826346.483018:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346292992:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826346.531000:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.531617:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332652160 00000100:00100000:3.0:1787826346.531628:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332652160 00000100:00100000:3.0:1787826346.531747:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68587 00000100:00100000:3.0:1787826346.531760:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226033:x1874663332652160:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826346.531781:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826346.531808:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bb3a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608018fc expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.531880:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bb41 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.531893:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bb41 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.531904:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bb41 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826346.532012:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e56:0x0]: returning size 4096 00000004:00200000:3.0:1787826346.532017:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787826346.532038:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bb41 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.532065:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bb3a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018fc expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.532087:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bb41 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018fc expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.532097:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794385a400) 00010000:00010000:3.0:1787826346.532100:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826346.532129:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826346.532194:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226033:x1874663332652160:12345-192.168.203.56@tcp:101:rm.0 Request processed in 438us (1202us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.532202:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68587 00000100:00080000:0.0:1787826346.535951:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826346.540038:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826346.540164:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332652288 00000100:00100000:0.0:1787826346.540175:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332652288 00000100:00100000:0.0:1787826346.540245:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68588 00000100:00100000:0.0:1787826346.540256:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226033:x1874663332652288:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826346.540274:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826346.540295:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4bb48 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801903 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.540363:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.540378:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.540393:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826346.540452:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x199:0x0]: returning size 4096 00000004:00200000:0.0:1787826346.540459:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826346.540496:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4bb48 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.540513:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.540521:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f784b4a5000) 00010000:00010000:0.0:1787826346.540524:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826346.540528:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826346.540669:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226033:x1874663332652288:12345-192.168.203.56@tcp:101:rm.0 Request processed in 413us (642us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826346.540686:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68588 00000100:00100000:3.0:1787826346.551068:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.551852:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332652416 00000100:00100000:2.0:1787826346.551864:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332652416 00000100:00100000:2.0:1787826346.551942:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68589 00000100:00100000:2.0:1787826346.551954:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226033:x1874663332652416:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826346.551981:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826346.552005:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bb56 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080190a expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.552067:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.552083:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.552094:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826346.552164:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x197:0x0]: returning size 4096 00000004:00200000:2.0:1787826346.552170:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826346.552211:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bb56 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.552230:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.552238:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826346.552241:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826346.552245:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826346.552313:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226033:x1874663332652416:12345-192.168.203.56@tcp:101:rm.0 Request processed in 361us (1253us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.552321:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68589 00000100:00080000:1.0:1787826346.552660:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826346.555929:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.556022:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332652672 00000100:00100000:3.0:1787826346.556032:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332652672 00000100:00100000:3.0:1787826346.556214:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68590 00000100:00100000:3.0:1787826346.556225:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226033:x1874663332652672:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826346.556245:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826346.556271:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bb64 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801911 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826346.556437:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e56:0x0]: returning size 4096 00010000:00010000:3.0:1787826346.556544:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bb64 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801911 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.556557:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7843326200) 00010000:00010000:3.0:1787826346.556560:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826346.556564:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bb64 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801911 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.556598:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826346.556661:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226033:x1874663332652672:12345-192.168.203.56@tcp:101:rm.0 Request processed in 438us (741us total) trans 4294985451 rc 301/301 00000100:00100000:3.0:1787826346.558278:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68590 00000100:00100000:1.0:1787826346.578835:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.578922:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332652800 00000100:00100000:3.0:1787826346.578932:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332652800 00000100:00100000:3.0:1787826346.579040:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9376 00000100:00100000:3.0:1787826346.579052:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226033:x1874663332652800:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826346.587335:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226033:x1874663332652800:12345-192.168.203.56@tcp:37:rm.0 Request processed in 8276us (8500us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.587349:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9376 00000100:00100000:3.0:1787826346.595378:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826346.595795:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332652928 00000100:00100000:0.0:1787826346.595806:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332652928 00000100:00100000:0.0:1787826346.595867:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9377 00000100:00100000:0.0:1787826346.595877:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226033:x1874663332652928:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826346.599236:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226033:x1874663332652928:12345-192.168.203.56@tcp:37:rm.0 Request processed in 3349us (3857us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.599249:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9377 00000100:00100000:3.0:1787826346.609491:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826346.609798:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332653184 00000100:00100000:0.0:1787826346.609809:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332653184 00000100:00100000:0.0:1787826346.609875:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9378 00000100:00100000:0.0:1787826346.609886:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943031050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226033:x1874663332653184:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:0.0:1787826346.610174:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1219, 344 off1111312 00000040:00080000:0.0:1787826346.610226:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1538, 424 off1555752 00000004:00080000:0.0:1787826346.610232:0:17601:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7845d45900 with version 6868 00000004:00080000:1.0:1787826346.611094:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6868 rpc_version 6868 00000020:00100000:1.0:1787826346.611840:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826346.611854:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784a91ea00 x1874663346293248/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826346.611880:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91ea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346293248:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826346.611955:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826346.612042:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826346.612429:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346293248 00000100:00100000:0.0:1787826346.612437:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346293248 00000100:00100000:0.0:1787826346.612494:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23966 00000100:00100000:0.0:1787826346.612504:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346293248:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826346.612877:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346293248:12345-0@lo:1000:osp_up0-1.0 Request processed in 373us (920us total) trans 8589957452 rc 0/0 00000100:00100000:0.0:1787826346.612885:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23966 00000100:00100000:1.0:1787826346.612944:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957448 gen 2 00000004:00080000:1.0:1787826346.612951:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957445 committed transno 8589957448 00000004:00080000:1.0:1787826346.612980:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957446 committed transno 8589957448 00000004:00080000:1.0:1787826346.612994:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957447 committed transno 8589957448 00000004:00080000:1.0:1787826346.613005:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957448 committed transno 8589957448 00000100:00100000:1.0:1787826346.613024:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f784a91ea00 x1874663346293248/t8589957452(8589957452) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826362 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826346.613093:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91ea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346293248:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826346.613379:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943031050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226033:x1874663332653184:12345-192.168.203.56@tcp:35:rm.0 Request processed in 3491us (3895us total) trans 4294985452 rc 0/0 00000100:00100000:0.0:1787826346.613388:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9378 00000100:00100000:1.0:1787826346.618454:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.621839:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332653440 00000100:00100000:3.0:1787826346.621853:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332653440 00000100:00100000:3.0:1787826346.621934:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68591 00000100:00100000:3.0:1787826346.621946:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226033:x1874663332653440:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826346.622463:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826346.622491:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bb72 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801918 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.622651:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bb79 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.622664:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bb79 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.622809:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bb79 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826346.622822:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e56:0x0]: returning size 4096 00010000:00010000:3.0:1787826346.622914:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bb72 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801918 expref: 13 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.622936:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bb79 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801918 expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.622945:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794385a400) 00010000:00010000:3.0:1787826346.622948:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1787826346.622952:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:3.0:1787826346.624900:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226033:x1874663332653440:12345-192.168.203.56@tcp:101:rm.0 Request processed in 2946us (6445us total) trans 4294985453 rc 0/0 00000100:00100000:3.0:1787826346.624913:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68591 00000100:00100000:1.0:1787826346.629224:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.629563:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332653568 00000100:00100000:2.0:1787826346.629586:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332653568 00000100:00100000:2.0:1787826346.629659:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68592 00000100:00100000:2.0:1787826346.629681:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226033:x1874663332653568:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:2.0:1787826346.629734:0:11629:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345647616 00000100:00100000:2.0:1787826346.629807:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226033:x1874663332653568:12345-192.168.203.56@tcp:41:rm.0 Request processed in 139us (593us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.629817:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68592 00000100:00100000:2.0:1787826346.634215:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826346.634296:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332653696 00000100:00100000:0.0:1787826346.634306:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332653696 00000100:00100000:0.0:1787826346.634369:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68593 00000100:00100000:0.0:1787826346.634381:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226033:x1874663332653696:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826346.634411:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826346.634433:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4bb87 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080191f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634500:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634513:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634524:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634543:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634550:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634555:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634633:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4bb9c lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634641:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4bb9c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634647:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4bb9c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826346.634830:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4bb9c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634844:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634850:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634855:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634862:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634869:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826346.634874:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634881:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634893:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:0.0:1787826346.634895:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826346.634901:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 3/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634907:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4bb8e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634920:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634926:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634931:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634936:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826346.634939:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634944:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634955:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f784b4a5b00) 00010000:00010000:0.0:1787826346.634957:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826346.634960:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.634966:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bb95 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.635002:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4bb87 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080191f expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.635016:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4bb9c lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080191f expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.635022:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826346.635025:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826346.635027:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826346.635179:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226033:x1874663332653696:12345-192.168.203.56@tcp:101:rm.0 Request processed in 799us (971us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826346.635187:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68593 00000100:00100000:0.0:1787826346.646840:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.657381:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332653824 00000100:00100000:3.0:1787826346.657396:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332653824 00000100:00100000:3.0:1787826346.657484:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68594 00000100:00100000:3.0:1787826346.657495:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226033:x1874663332653824:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826346.657525:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826346.657551:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bba3 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801926 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.657628:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bbaa lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.657646:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bbaa lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.657658:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bbaa lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826346.657720:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e57:0x0]: returning size 6 00000004:00200000:3.0:1787826346.657725:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:3.0:1787826346.657764:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bba3 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801926 expref: 14 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.657784:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bbaa lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801926 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.657793:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7843326200) 00010000:00010000:3.0:1787826346.657796:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826346.657800:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826346.657866:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226033:x1874663332653824:12345-192.168.203.56@tcp:101:rm.0 Request processed in 373us (11036us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.657873:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68594 00000100:00100000:2.0:1787826346.661577:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.661639:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.661679:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332653952 00000100:00100000:2.0:1787826346.661688:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332653952 00000100:00100000:2.0:1787826346.661746:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11169 00000100:00100000:2.0:1787826346.661757:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332653952:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826346.661772:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826346.661798:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4bb9c lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080191f expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.661816:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826346.661827:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826346.661878:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332653952:12345-192.168.203.56@tcp:103:rm.0 Request processed in 122us (238us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.661885:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11169 00000100:00100000:3.0:1787826346.667148:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332654208 00000100:00100000:3.0:1787826346.667162:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332654208 00000100:00100000:3.0:1787826346.667256:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68595 00000100:00100000:3.0:1787826346.667274:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226033:x1874663332654208:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826346.667332:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826346.667363:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bbaa lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e57:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801926 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.667391:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7843326200) 00010000:00010000:3.0:1787826346.667413:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826346.667462:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.667470:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.667479:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.667502:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346293376:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787826346.667568:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.667600:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.667890:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346293376 00000100:00100000:2.0:1787826346.667896:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346293376 00000100:00100000:2.0:1787826346.667939:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23967 00000100:00100000:2.0:1787826346.667947:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6528:x1874663346293376:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826346.667960:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826346.667975:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bbb1 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.667997:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.668008:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bbb1 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.668031:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.668045:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 9 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.668056:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346293504:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826346.668133:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.672747:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346293504:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826346.672785:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbb1 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.672798:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826346.672801:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826346.672805:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbb1 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.672816:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826346.672869:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6528:x1874663346293376:12345-0@lo:101:mdt00_001.0 Request processed in 4922us (5300us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.672877:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23967 00000100:00100000:3.0:1787826346.672964:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346293376:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787826346.672981:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.672991:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.681943:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.682219:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332654336 00000100:00100000:3.0:1787826346.682229:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332654336 00010000:00010000:3.0:1787826346.682284:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 10 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.682322:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22022 00000100:00100000:3.0:1787826346.682334:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332654336:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826346.682349:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826346.682356:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 10 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682369:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 10 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682392:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4bb5d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x197:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080190a expref: 10 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682419:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:3.0:1787826346.682427:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbb1 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682437:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bbb1 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682446:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bbb1 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682482:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbb1 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.682498:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346293632:0@lo:105: 00000100:00100000:3.0:1787826346.682607:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.682648:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826346.682667:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346293632 00000100:00100000:3.0:1787826346.682672:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346293632 00000100:00100000:3.0:1787826346.682700:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7901 00000100:00100000:3.0:1787826346.682707:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346293632:12345-0@lo:105: 00010000:00010000:3.0:1787826346.682717:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682727:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682736:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682743:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682751:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682759:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682778:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69200) 00000100:00100000:3.0:1787826346.682812:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346293632:12345-0@lo:105: Request processed in 105us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.682818:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7901 00010000:00010000:3.0:1787826346.682844:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682856:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826346.682883:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x197:0x0] 00010000:00010000:3.0:1787826346.682915:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 3/0,1 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682923:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 3/0,1 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.682933:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 3/0,1 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.682947:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346293760:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787826346.682974:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.682995:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826346.683019:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346293632:0@lo:105: 00010000:00010000:3.0:1787826346.683033:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826346.683072:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332654336:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 740us (1138us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.683079:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22022 00000100:00100000:3.0:1787826346.683101:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346293760 00000100:00100000:3.0:1787826346.683128:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346293760 00000100:00100000:3.0:1787826346.683153:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23968 00000100:00100000:3.0:1787826346.683159:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6528:x1874663346293760:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787826346.683172:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826346.683184:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4bbc6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bbbf expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.683203:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4bbc6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bbbf expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.683214:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4bbc6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbbf expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.683220:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f7843326d00) 00010000:00010000:3.0:1787826346.683223:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826346.683225:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68800, rc 0) 00000100:00100000:3.0:1787826346.683255:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346293760:12345-0@lo:101:mdt00_001.0 Request processed in 95us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.683260:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23968 00000100:00100000:3.0:1787826346.683288:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346293760:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787826346.683298:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbc6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.683307:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbc6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826346.683321:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x197:0x0] 00000100:00100000:3.0:1787826346.683351:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346293888:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787826346.683376:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.683389:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.683685:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346293888 00000100:00100000:2.0:1787826346.683690:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346293888 00000100:00100000:2.0:1787826346.683728:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23969 00000100:00100000:2.0:1787826346.683735:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346293888:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826346.683802:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346293888:12345-0@lo:1000:mdt00_001.0 Request processed in 67us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.683809:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23969 00000100:00100000:2.0:1787826346.683837:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346293888:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826346.683888:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346294016:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826346.683914:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.683929:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.683941:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346294016 00000100:00100000:2.0:1787826346.683944:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346294016 00000100:00100000:2.0:1787826346.683965:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23970 00000100:00100000:2.0:1787826346.683971:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346294016:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826346.684028:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346294016:12345-0@lo:1000:mdt00_001.0 Request processed in 56us (112us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.684035:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23970 00000100:00100000:2.0:1787826346.684060:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346294016:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826346.684096:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346294144:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826346.684139:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.684152:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.684565:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346294144 00000100:00100000:2.0:1787826346.684578:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346294144 00000100:00100000:2.0:1787826346.684602:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23971 00000100:00100000:2.0:1787826346.684609:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346294144:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826346.684686:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346294144:12345-0@lo:1000:mdt00_001.0 Request processed in 77us (546us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826346.684692:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23971 00000100:00100000:2.0:1787826346.684719:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346294144:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787826346.684769:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.684779:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.684787:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.684799:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346294272:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787826346.684822:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.684838:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.684849:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346294272 00000100:00100000:2.0:1787826346.684852:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346294272 00000100:00100000:2.0:1787826346.684874:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23972 00000100:00100000:2.0:1787826346.684880:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346294272:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826346.684887:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826346.684901:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bbd4 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bbcd expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.684921:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bbd4 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bbcd expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.684932:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bbd4 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbcd expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.684940:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7967ac8a00) 00010000:00010000:2.0:1787826346.684943:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826346.684945:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826346.684973:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6528:x1874663346294272:12345-0@lo:101:mdt00_001.0 Request processed in 93us (150us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.684978:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23972 00000100:00100000:2.0:1787826346.685002:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346294272:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826346.685013:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685023:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685045:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685053:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685060:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787826346.685388:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].229, 64 off22848 00000004:00080000:2.0:1787826346.685400:0:6528:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8158897293830ns, next at 8159897291412ns, rc = 0 00000040:00080000:2.0:1787826346.685444:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1220, 384 off1111696 00000040:00080000:2.0:1787826346.685487:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1539, 464 off1556216 00000004:00080000:2.0:1787826346.685492:0:6528:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784a358c00 with version 6869 00000004:00080000:2.0:1787826346.685511:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6869 rpc_version 6869 00000020:00100000:2.0:1787826346.685528:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826346.685535:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c97480 x1874663346294528/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826346.685549:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346294528:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826346.685583:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.685600:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.685616:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346294528 00000100:00100000:2.0:1787826346.685621:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346294528 00000100:00100000:2.0:1787826346.685645:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23973 00000100:00100000:2.0:1787826346.685653:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346294528:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826346.685872:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346294528:12345-0@lo:1000:osp_up0-1.0 Request processed in 219us (288us total) trans 8589957453 rc 0/0 00000100:00100000:2.0:1787826346.685879:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23973 00000100:00100000:2.0:1787826346.685918:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346294528:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826346.685948:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685959:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685968:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685974:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685981:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.685992:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686011:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787826346.686030:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686036:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686042:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686048:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826346.686053:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686060:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686070:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7967ac8e00) 00010000:00010000:2.0:1787826346.686072:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826346.686076:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686081:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbdb lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686093:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686126:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686134:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686147:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826346.686156:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x197:0x0] 00010000:00010000:2.0:1787826346.686162:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686169:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686177:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686187:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:2.0:1787826346.686190:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x197:0x0] 00010000:00010000:2.0:1787826346.686194:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbc6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686200:0:6528:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbc6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.686254:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226033:x1874663332654208:12345-192.168.203.56@tcp:36:rm.0 Request processed in 18981us (24685us total) trans 4294985454 rc 0/0 00000100:00100000:2.0:1787826346.686265:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68595 00010000:00010000:2.0:1787826346.686286:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1787826346.686291:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686298:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826346.686301:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686307:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686315:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7967ac8e00) 00010000:00010000:2.0:1787826346.686318:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826346.686321:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686326:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4bbe2 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e57:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686339:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826346.686342:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686347:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826346.686351:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826346.686359:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686381:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686387:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826346.686419:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686427:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbcd lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbd4 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686438:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826346.686441:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686447:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826346.686450:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826346.686457:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686472:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686477:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826346.686482:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686492:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69200/0xe21309aa81f4bbb1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbb8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686504:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826346.686506:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826346.686507:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826346.686514:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826346.686520:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826346.686530:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346294656:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826346.686565:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.686589:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826346.686595:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346294784:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826346.686617:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.686643:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346294656 00000100:00100000:2.0:1787826346.686648:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346294656 00000100:00100000:2.0:1787826346.686678:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346294784 00000100:00100000:2.0:1787826346.686681:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346294784 00000100:00100000:2.0:1787826346.686701:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11170 00000100:00100000:2.0:1787826346.686708:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346294656:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826346.686718:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826346.686733:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bbd4 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e57:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bbcd expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686743:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e57:0x0].0x0 (ffff9f7967ac8a00) 00010000:00010000:2.0:1787826346.686750:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826346.686777:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346294656:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (212us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.686786:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11170 00000100:00100000:2.0:1787826346.686800:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11171 00000100:00100000:2.0:1787826346.686806:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f794b470450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346294784:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826346.686814:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826346.686824:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbb8 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x197:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bbb1 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.686834:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826346.686840:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826346.686859:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f794b470450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346294784:12345-0@lo:103:ldlm_bl.0 Request processed in 54us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.686865:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11171 00000100:00100000:2.0:1787826346.686926:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346294784:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826346.686935:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346294656:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826346.697319:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.697416:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.697587:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332654720 00000100:00100000:3.0:1787826346.697597:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332654720 00000100:00100000:3.0:1787826346.697658:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9379 00000100:00100000:3.0:1787826346.697669:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226033:x1874663332654720:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:3.0:1787826346.697889:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:226033:x1874663332654720:12345-192.168.203.56@tcp:35:rm.0 Request processed in 221us (580us total) trans 4294985455 rc 0/0 00000100:00100000:3.0:1787826346.697897:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9379 00000100:00100000:0.0:1787826346.700724:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332654464 00000100:00100000:0.0:1787826346.700742:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332654464 00000100:00100000:0.0:1787826346.700821:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11172 00000100:00100000:0.0:1787826346.700836:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332654464:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826346.700851:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826346.700878:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4ba0d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018a1 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.700898:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f7843326b00) 00010000:00010000:0.0:1787826346.700903:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826346.700962:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332654464:12345-192.168.203.56@tcp:103:rm.0 Request processed in 132us (3546us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826346.700969:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11172 00000100:00100000:2.0:1787826346.715946:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.715983:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332654976 00000100:00100000:2.0:1787826346.715992:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332654976 00000100:00100000:2.0:1787826346.716046:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68596 00000100:00100000:2.0:1787826346.716057:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:226033:x1874663332654976:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:2.0:1787826346.716128:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826346.716155:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bb79 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e56:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801918 expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.716181:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bb41 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018fc expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.716194:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4ba1b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018a8 expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.716205:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794385a400) 00010000:00010000:2.0:1787826346.716210:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826346.716255:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.719413:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.719431:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.719461:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f480 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346294912:0@lo:101:mdt00_011.0 00000100:00100000:2.0:1787826346.719541:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.720732:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.720766:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346294912 00000100:00100000:2.0:1787826346.720775:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346294912 00000100:00100000:2.0:1787826346.720829:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23974 00000100:00100000:2.0:1787826346.720839:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346294912:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826346.720860:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826346.720881:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bbe9 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.720914:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.720924:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.720933:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bbe9 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.720964:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.720978:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 7 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.720991:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346295040:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826346.721073:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.721082:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 7 pid: 35080 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.721090:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346295168:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826346.721134:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.724837:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.724866:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.724893:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332655232 00000100:00100000:2.0:1787826346.724901:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332655232 00010000:00010000:2.0:1787826346.724942:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 8 pid: 35080 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.724971:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332655104 00000100:00100000:2.0:1787826346.724975:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332655104 00010000:00010000:2.0:1787826346.724987:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 8 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.725010:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22023 00000100:00100000:2.0:1787826346.725019:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332655232:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826346.725033:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826346.725040:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 8 pid: 35080 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.725050:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 8 pid: 35080 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.725070:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826346.725075:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bbe9 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.725083:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826346.725154:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332655232:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 136us (327us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.725160:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22023 00000100:00100000:2.0:1787826346.725179:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22024 00000100:00100000:2.0:1787826346.725186:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332655104:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826346.725193:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826346.725198:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 8 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.725207:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 8 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.725217:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826346.725221:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bbe9 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.725229:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4bbe9 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.725237:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4bbe9 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.729366:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbe9 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.729392:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346295296:0@lo:105: 00000100:00100000:2.0:1787826346.729475:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.729533:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.730439:0:6509:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787826346.730458:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346295296 00000100:00100000:2.0:1787826346.730466:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346295296 00000100:00100000:2.0:1787826346.730521:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7902 00000100:00100000:2.0:1787826346.730531:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e7a8c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346295296:12345-0@lo:105: 00010000:00010000:2.0:1787826346.730544:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.730556:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.730566:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.730593:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.730602:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.730612:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.730628:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eec00) 00000100:00100000:2.0:1787826346.730688:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e7a8c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346295296:12345-0@lo:105: Request processed in 158us (1212us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.730695:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7902 00000100:00100000:2.0:1787826346.730771:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346295296:0@lo:105: 00010000:00010000:2.0:1787826346.730789:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826346.730835:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332655104:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 5647us (5966us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.730844:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22024 00010000:00010000:2.0:1787826346.730888:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4ba5a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018c4 expref: 7 pid: 35094 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.730903:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346295040:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826346.730916:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4b9f1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080189a expref: 7 pid: 35080 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.730927:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346295168:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826346.730947:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbe9 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.730956:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826346.730959:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826346.730962:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826346.730996:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346294912:12345-0@lo:101:mdt00_011.0 Request processed in 10158us (11454us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.731002:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23974 00000100:00100000:2.0:1787826346.731036:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f480 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346294912:0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826346.731050:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826346.731070:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826346.731120:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.731133:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.731142:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.731159:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346295424:0@lo:101:mdt00_011.0 00000100:00100000:2.0:1787826346.731186:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.731206:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.731334:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346295424 00000100:00100000:2.0:1787826346.731338:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346295424 00000100:00100000:2.0:1787826346.731368:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23975 00000100:00100000:2.0:1787826346.731376:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346295424:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826346.731391:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826346.731416:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbfe lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bbf7 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.731435:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbfe lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bbf7 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.731445:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbfe lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbf7 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.731452:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8a00) 00010000:00010000:2.0:1787826346.731455:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826346.731457:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826346.731486:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346295424:12345-0@lo:101:mdt00_011.0 Request processed in 111us (300us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.731493:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23975 00000100:00100000:2.0:1787826346.731521:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346295424:0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826346.731530:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.731539:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826346.731552:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826346.731589:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346295552:0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826346.731614:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.731629:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.731644:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346295552 00000100:00100000:2.0:1787826346.731648:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346295552 00000100:00100000:2.0:1787826346.731669:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23976 00000100:00100000:2.0:1787826346.731675:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346295552:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826346.731738:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346295552:12345-0@lo:1000:mdt00_011.0 Request processed in 63us (123us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.731745:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23976 00000100:00100000:2.0:1787826346.731768:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346295552:0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826346.731804:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f800 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346295680:0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826346.731828:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.731840:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.731852:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346295680 00000100:00100000:2.0:1787826346.731855:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346295680 00000100:00100000:2.0:1787826346.731874:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23977 00000100:00100000:2.0:1787826346.731880:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346295680:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826346.731965:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346295680:12345-0@lo:1000:mdt00_011.0 Request processed in 84us (136us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826346.731971:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23977 00000100:00100000:2.0:1787826346.731997:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f800 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346295680:0@lo:1000:mdt00_011.0 00010000:00010000:2.0:1787826346.732032:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.732040:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.732048:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.732060:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f800 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346295808:0@lo:101:mdt00_011.0 00000100:00100000:2.0:1787826346.732082:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.732095:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.732126:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346295808 00000100:00100000:2.0:1787826346.732130:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346295808 00000100:00100000:2.0:1787826346.732151:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23978 00000100:00100000:2.0:1787826346.732157:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346295808:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826346.732164:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826346.732176:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bc05 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.732191:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.732204:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bc05 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.732226:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.732238:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 6 pid: 8752 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.732249:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c97800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346295936:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826346.732298:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.738827:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c97800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346295936:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826346.739175:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc05 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.739190:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f7843326b00) 00010000:00010000:2.0:1787826346.739193:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826346.739197:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc05 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.739207:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826346.739501:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346295808:12345-0@lo:101:mdt00_011.0 Request processed in 7342us (7416us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.739511:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23978 00000100:00100000:3.0:1787826346.743155:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f800 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346295808:0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826346.743184:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.743197:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.745604:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826346.747426:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332655360 00000100:00100000:0.0:1787826346.747439:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332655360 00010000:00010000:0.0:1787826346.747486:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 7 pid: 8752 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826346.747522:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22025 00000100:00100000:0.0:1787826346.747532:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332655360:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826346.747548:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826346.747555:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 7 pid: 8752 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.747567:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 7 pid: 8752 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.747603:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4ba76 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608018cb expref: 7 pid: 8752 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.747621:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f7843326b00) 00010000:00010000:0.0:1787826346.747627:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc05 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.747644:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bc05 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.747653:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bc05 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.748051:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc05 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826346.748068:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346296064:0@lo:105: 00000100:00100000:0.0:1787826346.748155:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826346.748198:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826346.749867:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346296064 00000100:00100000:3.0:1787826346.749880:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346296064 00000100:00100000:3.0:1787826346.749943:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7903 00000100:00100000:3.0:1787826346.749954:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346296064:12345-0@lo:105: 00010000:00010000:3.0:1787826346.749968:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.749980:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.749989:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.749998:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750008:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750017:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750041:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef400) 00000100:00100000:3.0:1787826346.750123:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346296064:12345-0@lo:105: Request processed in 171us (1966us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.750130:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7903 00010000:00010000:3.0:1787826346.750155:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750164:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750205:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750216:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750224:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750243:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,1 mode: --/EX res: [0x240000400:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750264:0:35094:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750273:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 4/0,1 mode: --/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750302:0:35094:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.750315:0:35094:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 11 pid: 6529 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.750329:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301c00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001_UUID:35094:1874663346296192:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826346.750413:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826346.756731:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346296064:0@lo:105: 00010000:00010000:0.0:1787826346.756760:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826346.756821:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332655360:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 9286us (11221us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826346.756832:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22025 00000100:00100000:3.0:1787826346.761512:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301c00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001_UUID:35094:1874663346296192:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826346.761550:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,1 mode: --/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.763850:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.766057:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332655488 00000100:00100000:3.0:1787826346.766071:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332655488 00010000:00010000:3.0:1787826346.772470:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 12 pid: 6529 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.772539:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22026 00000100:00100000:3.0:1787826346.772556:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332655488:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826346.772580:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826346.772587:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 12 pid: 6529 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.772600:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 12 pid: 6529 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.772633:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bb4f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801903 expref: 12 pid: 6529 timeout: 8258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.772646:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f784b4a5000) 00010000:00010000:3.0:1787826346.772651:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,1 mode: --/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.772659:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.772823:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.772853:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826346.772926:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332655488:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 377us (9084us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.772934:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22026 00010000:00010000:3.0:1787826346.772966:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.772974:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.773019:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.773029:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.773038:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.773058:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346296320:0@lo:101:mdt00_011.0 00000100:00100000:3.0:1787826346.773130:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.773158:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826346.773174:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346296320 00000100:00100000:3.0:1787826346.773180:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346296320 00000100:00100000:3.0:1787826346.773323:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23979 00000100:00100000:3.0:1787826346.773331:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346296320:12345-0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826346.773345:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826346.773360:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bc28 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x197:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bc21 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.773379:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bc28 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bc21 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.773390:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bc28 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc21 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.773397:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7843326400) 00010000:00010000:3.0:1787826346.773400:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826346.773415:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826346.773445:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346296320:12345-0@lo:101:mdt00_011.0 Request processed in 116us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.773454:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23979 00000100:00100000:3.0:1787826346.773494:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346296320:0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826346.773507:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.773517:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.776076:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346296448:0@lo:403:mdt00_011.0 00000100:00100000:3.0:1787826346.776283:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.776681:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.776728:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346296448 00000100:00100000:2.0:1787826346.776736:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346296448 00000100:00100000:2.0:1787826346.776827:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23980 00000100:00100000:2.0:1787826346.776838:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346296448:12345-0@lo:403:mdt00_011.0 00000100:00100000:2.0:1787826346.778665:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346296448:12345-0@lo:403:mdt00_011.0 Request processed in 1821us (2374us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.778678:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23980 00000100:00100000:3.0:1787826346.778824:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346296448:0@lo:403:mdt00_011.0 00000040:00080000:3.0:1787826346.779827:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1221, 952 off1112648 00000040:00080000:3.0:1787826346.780013:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1540, 1032 off1557248 00000004:00080000:3.0:1787826346.780020:0:35094:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7850703a80 with version 6870 00000004:00080000:3.0:1787826346.780047:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6870 rpc_version 6870 00000020:00100000:3.0:1787826346.780076:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826346.780086:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7848303b80 x1874663346296704/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826346.780132:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346296704:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826346.780190:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.781149:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826346.781232:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346296704 00000100:00100000:0.0:1787826346.781241:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346296704 00000100:00100000:2.0:1787826346.781317:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23981 00000100:00100000:2.0:1787826346.781329:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346296704:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826346.781861:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346296704:12345-0@lo:1000:osp_up0-1.0 Request processed in 533us (1669us total) trans 8589957454 rc 0/0 00000100:00100000:2.0:1787826346.781869:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23981 00000100:00100000:3.0:1787826346.782248:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848303b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346296704:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787826346.782457:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e56:0x0]: returning size 4096 00010000:00010000:1.0:1787826346.782483:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782493:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782501:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782507:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826346.782512:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782520:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782536:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x199:0x0].0x0 (ffff9f784b4a5000) 00010000:00010000:1.0:1787826346.782539:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826346.782544:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 2/0,0 mode: --/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782550:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4bc1a lrc: 0/0,0 mode: --/EX res: [0x240000400:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782571:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782580:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782587:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782615:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826346.782624:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x197:0x0] 00010000:00010000:1.0:1787826346.782632:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782639:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782645:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782650:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826346.782653:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782659:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782667:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794385a400) 00010000:00010000:1.0:1787826346.782669:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826346.782672:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782678:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bc13 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782687:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782695:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782701:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782717:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787826346.782728:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826346.782733:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782739:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782747:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782759:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787826346.782762:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826346.782766:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.782773:0:35094:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826346.782848:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226033:x1874663332654976:12345-192.168.203.56@tcp:36:rm.0 Request processed in 66791us (66909us total) trans 4294985456 rc 0/0 00000100:00100000:1.0:1787826346.782861:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68596 00010000:00010000:3.0:1787826346.782932:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826346.782939:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.782946:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826346.782951:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826346.782959:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.782991:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.782998:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826346.783031:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783039:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68000/0xe21309aa81f4bc21 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc28 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783053:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826346.783057:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783064:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826346.783067:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826346.783074:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783097:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783130:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826346.783137:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783145:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef400/0xe21309aa81f4bc05 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc0c expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783157:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826346.783160:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783167:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826346.783170:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826346.783178:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783197:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783203:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826346.783218:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783230:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eec00/0xe21309aa81f4bbe9 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbf0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783240:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826346.783242:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826346.783244:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826346.783258:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346296960:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826346.783304:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.783334:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826346.783340:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346296832:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826346.783363:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.783388:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346297088:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826346.783427:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.783461:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346296960 00000100:00100000:3.0:1787826346.783467:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346296960 00000100:00100000:3.0:1787826346.783500:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346296832 00000100:00100000:3.0:1787826346.783503:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346296832 00000100:00100000:3.0:1787826346.783519:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346297088 00000100:00100000:3.0:1787826346.783522:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346297088 00000100:00100000:3.0:1787826346.783541:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11173 00000100:00100000:3.0:1787826346.783549:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+17:3655:x1874663346296960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826346.783561:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826346.783580:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bc0c lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e56:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bc05 expref: 17 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783592:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f7843326b00) 00010000:00010000:3.0:1787826346.783599:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826346.783626:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+16:3655:x1874663346296960:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (322us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.783632:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11173 00000100:00100000:3.0:1787826346.783648:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11174 00000100:00100000:3.0:1787826346.783654:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943ed7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+15:3655:x1874663346296832:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826346.783662:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826346.783673:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bc28 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x197:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bc21 expref: 15 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783682:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x0 (ffff9f7843326400) 00010000:00010000:3.0:1787826346.783688:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826346.783709:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943ed7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+14:3655:x1874663346296832:12345-0@lo:103:ldlm_bl.0 Request processed in 55us (345us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.783715:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11174 00000100:00100000:3.0:1787826346.783758:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11175 00000100:00100000:3.0:1787826346.783765:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+13:3652:x1874663346297088:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826346.783772:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826346.783782:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bbf0 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bbe9 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.783792:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826346.783796:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826346.783821:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346297088:12345-0@lo:103:ldlm_bl.0 Request processed in 57us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.783827:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11175 00000100:00100000:3.0:1787826346.783862:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346296832:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826346.783872:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346296960:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826346.783903:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346297088:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826346.785867:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826346.788327:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826346.839624:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826346.839851:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332655616 00000100:00100000:1.0:1787826346.839861:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332655616 00000100:00100000:1.0:1787826346.839934:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68597 00000100:00100000:1.0:1787826346.839946:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226041:x1874663332655616:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:1.0:1787826346.840003:0:35080:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345647616 00000100:00100000:1.0:1787826346.840081:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226041:x1874663332655616:12345-192.168.203.56@tcp:41:stat.0 Request processed in 136us (469us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826346.840088:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68597 00000100:00100000:3.0:1787826346.908700:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826346.908809:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332655744 00000100:00100000:2.0:1787826346.908818:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332655744 00000100:00100000:2.0:1787826346.908889:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68598 00000100:00100000:2.0:1787826346.908900:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226045:x1874663332655744:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826346.908921:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826346.908943:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc2f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080192d expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.909011:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.909025:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.909036:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826346.909090:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826346.909094:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826346.909157:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.909182:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc2f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.909213:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826346.909222:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826346.909225:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826346.909229:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efa00, rc 0) 00000100:00100000:2.0:1787826346.909417:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226045:x1874663332655744:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 517us (726us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.909425:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68598 00000100:00100000:2.0:1787826346.914293:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.914570:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332655872 00000100:00100000:3.0:1787826346.914579:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332655872 00000100:00100000:3.0:1787826346.914687:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68599 00000100:00100000:3.0:1787826346.914699:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226045:x1874663332655872:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826346.914720:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826346.914742:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bc3d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801934 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.914815:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.914830:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.914841:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.914852:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.914863:0:35092:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbfe lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bbf7 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.914877:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.914912:0:35092:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbfe lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4bbf7 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.914930:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346297216:0@lo:104: 00000100:00100000:3.0:1787826346.914990:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.915041:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826346.915205:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346297216 00000100:00100000:0.0:1787826346.915211:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346297216 00000100:00100000:0.0:1787826346.915251:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7904 00000100:00100000:0.0:1787826346.915258:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35092:x1874663346297216:12345-0@lo:104: 00010000:00010000:0.0:1787826346.915271:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.915342:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826346.915349:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35092:x1874663346297216:12345-0@lo:104: Request processed in 91us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826346.915355:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7904 00010000:00010000:0.0:1787826346.915378:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826346.915384:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.915392:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826346.915397:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826346.915418:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.915443:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.915453:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826346.915476:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.915483:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eee00/0xe21309aa81f4bbf7 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bbfe expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.915503:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826346.915505:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826346.915507:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826346.915518:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346297344:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826346.915549:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826346.915587:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346297344 00000100:00100000:0.0:1787826346.915591:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346297344 00010000:00010000:0.0:1787826346.915607:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbfe lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4bbf7 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826346.915629:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22027 00000100:00100000:0.0:1787826346.915635:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346297344:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826346.915644:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826346.916180:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346297216:0@lo:104: 00010000:00010000:1.0:1787826346.916203:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826346.921515:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957454 is committed 00010000:00010000:0.0:1787826346.921993:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bbfe lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4bbf7 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.922016:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8a00) 00010000:00010000:0.0:1787826346.922022:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.922032:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.922039:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826346.922144:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826346.922233:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346297344:12345-0@lo:103:ldlm_bl.0 Request processed in 6596us (6681us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826346.922242:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22027 00000100:00100000:3.0:1787826346.922571:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346297344:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826346.922871:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.922880:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923016:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923025:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923032:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923043:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923049:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826346.923053:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923059:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923071:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826346.923074:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826346.923078:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923228:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4bc44 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923244:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.923250:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.931587:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.931605:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826346.931611:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.931626:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.931646:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8a00) 00010000:00010000:1.0:1787826346.931648:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826346.931653:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.931660:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4bc4b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.931741:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bc3d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801934 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.931751:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826346.931753:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826346.931757:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bc3d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801934 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.931767:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:1.0:1787826346.932017:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226045:x1874663332655872:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 17317us (17740us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826346.932026:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68599 00000100:00100000:0.0:1787826346.948908:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.949093:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332656000 00000100:00100000:3.0:1787826346.949102:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332656000 00000100:00100000:3.0:1787826346.949191:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68600 00000100:00100000:3.0:1787826346.949203:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226045:x1874663332656000:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826346.949224:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826346.949244:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bc52 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080193b expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.949323:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.949337:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.949348:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826346.949412:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826346.949416:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826346.949436:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.949458:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bc52 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.949475:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.949482:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826346.949485:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826346.949488:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826346.949549:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226045:x1874663332656000:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 349us (652us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.949555:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68600 00000100:00100000:2.0:1787826346.967994:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826346.968081:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332656256 00000100:00100000:3.0:1787826346.968091:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332656256 00000100:00100000:3.0:1787826346.968179:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68601 00000100:00100000:3.0:1787826346.968190:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226045:x1874663332656256:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:3.0:1787826346.968319:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346297472:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826346.968382:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.968424:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826346.968552:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346297472 00000100:00100000:1.0:1787826346.968558:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346297472 00000100:00100000:1.0:1787826346.968588:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23982 00000100:00100000:1.0:1787826346.968595:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346297472:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787826346.968684:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346297472:12345-0@lo:1000:mdt00_000.0 Request processed in 88us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826346.968691:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23982 00000100:00100000:3.0:1787826346.969182:0:6527:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957454 gen 2 00000004:00080000:3.0:1787826346.969189:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957452 committed transno 8589957454 00000004:00080000:3.0:1787826346.969211:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957453 committed transno 8589957454 00000004:00080000:3.0:1787826346.969224:0:6527:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957454 committed transno 8589957454 00000100:00100000:3.0:1787826346.969245:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346297472:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826346.971122:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346297600:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826346.971194:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.971234:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826346.972415:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346297600 00000100:00100000:2.0:1787826346.972433:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346297600 00000100:00100000:2.0:1787826346.972504:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23983 00000100:00100000:2.0:1787826346.972513:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346297600:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826346.972659:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346297600:12345-0@lo:1000:mdt00_000.0 Request processed in 146us (1464us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826346.972666:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23983 00000100:00100000:3.0:1787826346.973858:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346297600:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1787826346.973941:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.973950:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.973960:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.973980:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346297728:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787826346.974029:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.974065:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826346.975441:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346297728 00000100:00100000:1.0:1787826346.975454:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346297728 00000100:00100000:1.0:1787826346.975526:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23984 00000100:00100000:1.0:1787826346.975536:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6527:x1874663346297728:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787826346.975554:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826346.975577:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bc60 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.975603:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 6 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.975611:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.975619:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.975649:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.975661:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 7 pid: 35093 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826346.975674:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346297856:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826346.975772:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 6 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.975781:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 6 pid: 24703 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826346.975790:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346297984:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826346.975815:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826346.987861:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346297856:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826346.987882:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346297984:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826346.987930:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.987943:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826346.987946:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826346.987951:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826346.987961:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826346.988041:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346297728:12345-0@lo:101:mdt00_000.0 Request processed in 12507us (14011us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826346.988049:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23984 00000100:00100000:3.0:1787826346.988640:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346297728:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787826346.988660:0:6527:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.988683:0:6527:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826346.988942:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826346.989704:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346298240:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826346.989754:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826346.989793:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826346.989799:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d008000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346298112:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826346.989822:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826346.990134:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346298240 00000100:00100000:1.0:1787826346.990142:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346298240 00000100:00100000:1.0:1787826346.990188:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346298112 00000100:00100000:1.0:1787826346.990191:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346298112 00000100:00100000:1.0:1787826346.990207:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6252 00000100:00100000:1.0:1787826346.990214:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346298240:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826346.990238:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826346.990290:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346298240:12345-0@lo:13:osp-pre-0-1.0 Request processed in 66us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826346.990297:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6252 00000100:00100000:1.0:1787826346.990314:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6253 00000100:00100000:1.0:1787826346.990320:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346298112:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826346.990330:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826346.990354:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346298112:12345-0@lo:13:osp-pre-0-0.0 Request processed in 34us (531us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826346.990360:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6253 00000100:00100000:1.0:1787826346.990418:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d008000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346298112:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826346.990432:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346298240:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826346.990629:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826346.990640:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346298368:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826346.990688:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826346.991172:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332656384 00000100:00100000:3.0:1787826346.991178:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332656384 00010000:00010000:3.0:1787826346.991207:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 8 pid: 35093 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826346.991237:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22028 00000100:00100000:3.0:1787826346.991245:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332656384:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826346.991256:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826346.991263:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 8 pid: 35093 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.991274:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 8 pid: 35093 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.991305:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc59 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080193b expref: 8 pid: 35093 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.991321:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826346.991327:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826346.991336:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826346.991382:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332656384:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 137us (2440us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826346.991389:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22028 00000100:00100000:2.0:1787826346.992607:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826346.992624:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346298624:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826346.992696:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826346.992740:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346298368 00000100:00100000:2.0:1787826346.992747:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346298368 00000100:00100000:2.0:1787826346.992783:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346298624 00000100:00100000:2.0:1787826346.992787:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346298624 00000100:00100000:2.0:1787826346.992807:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23985 00000100:00100000:2.0:1787826346.992815:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346298368:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826346.992844:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442766848 avail 1296527360 00000100:00100000:2.0:1787826346.992893:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346298368:12345-0@lo:41:osp-pre-0-1.0 Request processed in 78us (2203us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.992899:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23985 00000100:00100000:2.0:1787826346.992916:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23986 00000100:00100000:2.0:1787826346.992932:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346298624:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826346.992953:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443389440 avail 1297149952 00000100:00100000:2.0:1787826346.992983:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346298624:12345-0@lo:41:osp-pre-1-0.0 Request processed in 62us (288us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.992989:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23986 00000100:00100000:2.0:1787826346.993031:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346298624:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826346.994711:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346298496:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826346.994778:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826346.994861:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346298368:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826346.996422:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346298496 00000100:00100000:2.0:1787826346.996434:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346298496 00000100:00100000:2.0:1787826346.996501:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6254 00000100:00100000:2.0:1787826346.996511:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346298496:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826346.996542:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826346.996628:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346298496:12345-0@lo:13:osp-pre-1-0.0 Request processed in 118us (1849us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826346.996635:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6254 00000100:00100000:0.0:1787826346.997452:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346298496:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826347.008659:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826347.008956:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332656512 00000100:00100000:3.0:1787826347.008965:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332656512 00010000:00010000:3.0:1787826347.009026:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 7 pid: 24703 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826347.009075:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22029 00000100:00100000:3.0:1787826347.009087:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332656512:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826347.009101:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826347.009144:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 7 pid: 24703 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.009158:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 7 pid: 24703 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.009181:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4bc36 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080192d expref: 7 pid: 24703 timeout: 8259 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.009200:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826347.009207:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.009219:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.009228:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.009262:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826347.009278:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346298752:0@lo:105: 00000100:00100000:3.0:1787826347.009338:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.009548:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346298752 00000100:00100000:2.0:1787826347.009553:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346298752 00000100:00100000:2.0:1787826347.009584:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7905 00000100:00100000:2.0:1787826347.009592:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346298752:12345-0@lo:105: 00010000:00010000:2.0:1787826347.009602:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009612:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009620:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009627:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009633:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009641:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009676:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e69c00) 00000100:00100000:2.0:1787826347.009712:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346298752:12345-0@lo:105: Request processed in 121us (374us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826347.009722:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7905 00010000:00010000:2.0:1787826347.009749:0:6527:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009759:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826347.009784:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826347.009814:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009820:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.009829:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826347.009843:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346298880:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787826347.009884:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.009903:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826347.009920:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346298880 00000100:00100000:2.0:1787826347.009924:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346298880 00000100:00100000:2.0:1787826347.009952:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23987 00000100:00100000:2.0:1787826347.009959:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6527:x1874663346298880:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787826347.009973:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826347.009987:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc75 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bc6e expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.010004:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc75 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bc6e expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.010016:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc75 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc6e expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.010023:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8400) 00010000:00010000:2.0:1787826347.010026:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826347.010029:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826347.010058:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346298880:12345-0@lo:101:mdt00_000.0 Request processed in 100us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826347.010064:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23987 00000100:00100000:2.0:1787826347.010102:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346298880:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787826347.010141:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.010150:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826347.010161:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826347.010193:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299008:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010220:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.010236:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3580c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826347.010250:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346299008 00000100:00100000:2.0:1787826347.010254:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346299008 00000100:00100000:2.0:1787826347.010277:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23988 00000100:00100000:2.0:1787826347.010297:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299008:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010356:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299008:12345-0@lo:1000:mdt00_000.0 Request processed in 59us (135us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826347.010362:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23988 00000100:00100000:2.0:1787826347.010409:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299008:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010446:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299136:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010468:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.010482:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3580c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826347.010495:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346299136 00000100:00100000:2.0:1787826347.010498:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346299136 00000100:00100000:2.0:1787826347.010519:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23989 00000100:00100000:2.0:1787826347.010525:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299136:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010604:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299136:12345-0@lo:1000:mdt00_000.0 Request processed in 78us (134us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826347.010610:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23989 00000100:00100000:2.0:1787826347.010637:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299136:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010708:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299264:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010732:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.010746:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3580c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826347.010756:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346299264 00000100:00100000:2.0:1787826347.010759:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346299264 00000100:00100000:2.0:1787826347.010778:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23990 00000100:00100000:2.0:1787826347.010784:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299264:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010835:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299264:12345-0@lo:1000:mdt00_000.0 Request processed in 50us (102us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826347.010841:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23990 00000100:00100000:2.0:1787826347.010866:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299264:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010907:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299392:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.010927:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.010939:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3580c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826347.010950:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346299392 00000100:00100000:2.0:1787826347.010953:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346299392 00000100:00100000:2.0:1787826347.010975:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23991 00000100:00100000:2.0:1787826347.010980:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299392:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.011035:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299392:12345-0@lo:1000:mdt00_000.0 Request processed in 54us (107us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826347.011041:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23991 00000100:00100000:3.0:1787826347.015031:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826347.015081:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346298752:0@lo:105: 00010000:00010000:3.0:1787826347.015121:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826347.015192:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332656512:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 6107us (6541us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826347.015204:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22029 00000100:00100000:2.0:1787826347.016246:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299392:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787826347.016347:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299520:0@lo:1000:mdt00_000.0 00000100:00080000:1.0:1787826347.016469:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826347.017006:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.017053:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a3580c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826347.017386:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346299520 00000100:00100000:0.0:1787826347.017394:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346299520 00000100:00100000:0.0:1787826347.017460:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23992 00000100:00100000:0.0:1787826347.017470:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299520:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826347.017589:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299520:12345-0@lo:1000:mdt00_000.0 Request processed in 120us (583us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826347.017596:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23992 00000100:00100000:3.0:1787826347.017812:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299520:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826347.017888:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299648:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787826347.017930:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826347.017953:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826347.018158:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346299648 00000100:00100000:0.0:1787826347.018163:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346299648 00000100:00100000:0.0:1787826347.018200:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23993 00000100:00100000:0.0:1787826347.018208:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299648:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787826347.018264:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6527:x1874663346299648:12345-0@lo:1000:mdt00_000.0 Request processed in 57us (343us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826347.018271:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23993 00000100:00100000:3.0:1787826347.018533:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346299648:0@lo:1000:mdt00_000.0 00000040:00080000:3.0:1787826347.020821:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1223, 2168 off1116280 00000040:00080000:3.0:1787826347.021066:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1541, 2368 off1559616 00000004:00080000:3.0:1787826347.021074:0:6527:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d067180 with version 6871 00000004:00080000:2.0:1787826347.022839:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6871 rpc_version 6871 00000020:00100000:2.0:1787826347.022907:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826347.022918:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78508c2300 x1874663346299904/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826347.023073:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346299904:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826347.023192:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.023227:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826347.023324:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346299904 00000100:00100000:0.0:1787826347.023331:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346299904 00000100:00100000:0.0:1787826347.023413:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23994 00000100:00100000:0.0:1787826347.023423:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346299904:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826347.027446:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346299904:12345-0@lo:1000:osp_up0-1.0 Request processed in 4017us (4246us total) trans 8589957455 rc 0/0 00000100:00100000:0.0:1787826347.027460:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23994 00000100:00100000:2.0:1787826347.028052:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346299904:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787826347.028813:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.028830:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.028844:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.028998:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.029007:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.029015:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.029048:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.029056:0:6527:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.029066:0:6527:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826347.029089:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346300032:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787826347.029176:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826347.030024:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d0676c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826347.033014:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346300032 00000100:00100000:1.0:1787826347.033032:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346300032 00000100:00100000:1.0:1787826347.033194:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23995 00000100:00100000:1.0:1787826347.033205:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6527:x1874663346300032:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787826347.033230:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826347.033254:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4bc91 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bc8a expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.033297:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4bc91 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bc8a expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.033311:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4bc91 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc8a expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.033319:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f794385a300) 00010000:00010000:1.0:1787826347.033322:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826347.033326:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826347.033421:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6527:x1874663346300032:12345-0@lo:101:mdt00_000.0 Request processed in 219us (4244us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826347.033429:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23995 00000100:00100000:3.0:1787826347.033518:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6527:1874663346300032:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787826347.033539:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033550:0:6527:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826347.033586:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006b6e3ba9 reply state 000000000802dc34 transno 4294985457 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033599:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033607:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826347.033610:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826347.033613:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826347.033619:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033625:0:6527:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826347.033637:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00000004:00010000:3.0:1787826347.033646:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006b6e3ba9 reply state 000000000802dc34 transno 4294985457 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033654:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033660:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826347.033675:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826347.033677:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787826347.033682:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e58:0x0]: returning size 4096 00000004:00080000:3.0:1787826347.033691:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826347.033701:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033708:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033714:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033731:0:6527:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826347.033735:0:6527:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826347.033739:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.033745:0:6527:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826347.033817:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226045:x1874663332656256:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 65628us (65832us total) trans 4294985457 rc 0/0 00000100:00100000:3.0:1787826347.033824:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68601 00010000:00010000:0.0:1787826347.037383:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826347.037410:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.037424:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826347.037432:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826347.037442:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.037498:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.037505:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826347.037547:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.037554:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69c00/0xe21309aa81f4bc60 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bc67 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.037572:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826347.037573:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826347.037575:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826347.037583:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826347.037605:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346300160:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826347.037688:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826347.037741:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346300160 00000100:00100000:0.0:1787826347.037747:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346300160 00000100:00100000:0.0:1787826347.037799:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11176 00000100:00100000:0.0:1787826347.037808:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346300160:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826347.037820:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826347.037840:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4bc67 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bc60 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.037867:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826347.037872:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826347.037905:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346300160:12345-0@lo:103:ldlm_bl.0 Request processed in 99us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826347.037911:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11176 00000100:00100000:0.0:1787826347.037957:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346300160:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826347.052871:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826347.053093:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332656640 00000100:00100000:0.0:1787826347.053103:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332656640 00000100:00100000:0.0:1787826347.053206:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68602 00000100:00100000:0.0:1787826347.053217:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226034:x1874663332656640:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826347.053237:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826347.053259:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bc98 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801942 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.053339:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bc9f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.053354:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bc9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.053363:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bc9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826347.053428:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826347.053432:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826347.053453:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bc9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.053479:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bc98 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801942 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.053497:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bc9f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801942 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.053505:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826347.053508:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826347.053512:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826347.053572:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226034:x1874663332656640:12345-192.168.203.56@tcp:101:bash.0 Request processed in 357us (720us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826347.053579:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68602 00000100:00100000:3.0:1787826347.062875:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826347.063148:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332656768 00000100:00100000:1.0:1787826347.063163:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332656768 00000100:00100000:1.0:1787826347.063230:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68603 00000100:00100000:1.0:1787826347.063241:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226034:x1874663332656768:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826347.063261:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826347.063284:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4bca6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801949 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.063350:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.063362:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.063372:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.063383:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.063392:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc75 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc6e expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.063411:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.063445:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc75 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4bc6e expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826347.063459:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346300288:0@lo:104: 00000100:00100000:1.0:1787826347.063506:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826347.063543:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826347.063617:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346300288 00000100:00100000:2.0:1787826347.063622:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346300288 00000100:00100000:2.0:1787826347.063665:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7906 00000100:00100000:2.0:1787826347.063682:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346300288:12345-0@lo:104: 00010000:00010000:2.0:1787826347.063695:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.063758:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826347.063764:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346300288:12345-0@lo:104: Request processed in 84us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826347.063770:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7906 00010000:00010000:2.0:1787826347.063792:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826347.063798:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.063806:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826347.063811:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826347.063819:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.063857:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.063864:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826347.063886:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.063893:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee400/0xe21309aa81f4bc6e lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bc75 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.063916:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826347.063925:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826347.063927:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826347.063938:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346300416:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826347.063968:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826347.064001:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346300416 00000100:00100000:2.0:1787826347.064005:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346300416 00010000:00010000:2.0:1787826347.064020:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc75 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4bc6e expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826347.064042:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22030 00000100:00100000:2.0:1787826347.064047:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346300416:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826347.064057:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826347.065343:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346300288:0@lo:104: 00010000:00010000:1.0:1787826347.065371:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826347.067787:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957455 is committed 00010000:00010000:0.0:1787826347.069007:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4bc75 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4bc6e expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.069034:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8400) 00010000:00010000:0.0:1787826347.069040:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.069049:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.069056:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.069804:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826347.069912:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3652:x1874663346300416:12345-0@lo:103:ldlm_bl.0 Request processed in 5861us (5940us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826347.069933:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22030 00000100:00100000:2.0:1787826347.070170:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346300416:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826347.070385:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.070394:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826347.070541:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:8752:1874663346300544:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826347.070649:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826347.070771:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826347.071554:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346300544 00000100:00100000:2.0:1787826347.071565:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346300544 00000100:00100000:2.0:1787826347.071775:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23996 00000100:00100000:2.0:1787826347.071787:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:8752:x1874663346300544:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787826347.071905:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:8752:x1874663346300544:12345-0@lo:1000:mdt00_003.0 Request processed in 119us (1255us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826347.071913:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23996 00000100:00100000:1.0:1787826347.072102:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:8752:1874663346300544:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1787826347.072190:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bcbb lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072205:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bcbb lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072216:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bcbb lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826347.072240:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bcbb lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072255:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072262:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072268:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072274:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072280:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826347.072284:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072291:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072303:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826347.072307:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826347.072311:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072317:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bcad lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072332:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072338:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072344:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072349:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826347.072352:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072358:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072365:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac8400) 00010000:00010000:1.0:1787826347.072368:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826347.072370:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072376:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bcb4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072428:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4bca6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801949 expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072447:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bcbb lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801949 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.072454:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f794385ae00) 00010000:00010000:1.0:1787826347.072457:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826347.072459:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826347.072525:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226034:x1874663332656768:12345-192.168.203.56@tcp:101:bash.0 Request processed in 9286us (9660us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826347.072532:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68603 00000100:00100000:1.0:1787826347.076379:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826347.076472:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332656896 00000100:00100000:3.0:1787826347.076482:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332656896 00000100:00100000:3.0:1787826347.076557:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68604 00000100:00100000:3.0:1787826347.076579:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226034:x1874663332656896:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826347.076605:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826347.076626:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bcc2 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801950 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.076696:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4bcc9 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.076709:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4bcc9 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.076720:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4bcc9 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826347.076811:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e58:0x0]: returning size 4096 00000004:00200000:3.0:1787826347.076816:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826347.076864:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bcc2 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801950 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.076882:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4bcc9 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801950 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.076890:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826347.076893:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826347.076897:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826347.077165:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226034:x1874663332656896:12345-192.168.203.56@tcp:101:bash.0 Request processed in 598us (795us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826347.077173:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68604 00000100:00080000:1.0:1787826347.079488:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826347.080265:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826347.080397:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332657024 00000100:00100000:1.0:1787826347.080417:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332657024 00000100:00100000:1.0:1787826347.080484:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68605 00000100:00100000:1.0:1787826347.080495:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226034:x1874663332657024:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826347.080515:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826347.080537:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4bcd0 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801957 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.080606:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bcd7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.080619:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bcd7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.080629:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bcd7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826347.080693:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19a:0x0]: returning size 4096 00000004:00200000:1.0:1787826347.080699:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826347.080735:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4bcd0 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801957 expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.080754:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bcd7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801957 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.080763:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:1.0:1787826347.080767:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826347.080771:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826347.080828:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226034:x1874663332657024:12345-192.168.203.56@tcp:101:bash.0 Request processed in 335us (569us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826347.080836:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68605 00000100:00100000:3.0:1787826347.084007:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826347.084262:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332657152 00000100:00100000:0.0:1787826347.084272:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332657152 00000100:00100000:0.0:1787826347.084333:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68606 00000100:00100000:0.0:1787826347.084345:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226034:x1874663332657152:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826347.084364:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826347.084385:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bcde lrc: 2/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080195e expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.084456:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.084470:0:11629:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4bc91 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bc8a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.084479:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.084516:0:11629:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4bc91 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4bc8a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826347.084533:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346300672:0@lo:104: 00000100:00100000:0.0:1787826347.084586:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826347.084628:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826347.084791:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346300672 00000100:00100000:3.0:1787826347.084796:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346300672 00000100:00100000:3.0:1787826347.084838:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7907 00000100:00100000:3.0:1787826347.084857:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346300672:12345-0@lo:104: 00010000:00010000:3.0:1787826347.084870:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.084972:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826347.084979:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346300672:12345-0@lo:104: Request processed in 134us (393us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826347.084986:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7907 00010000:00010000:3.0:1787826347.085609:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826347.085617:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.085630:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826347.085636:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826347.085645:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.085685:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.085692:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826347.085722:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 1/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.085730:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4bc8a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bc91 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.085750:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826347.085752:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826347.085753:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826347.085760:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826347.085773:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346300800:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826347.085819:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826347.085867:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346300800 00000100:00100000:3.0:1787826347.085872:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346300800 00010000:00010000:3.0:1787826347.085894:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4bc91 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4bc8a expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826347.086092:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22031 00000100:00100000:3.0:1787826347.086102:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7947dfcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3652:x1874663346300800:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826347.086139:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826347.087040:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f794385a300) 00010000:00010000:3.0:1787826347.087049:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.087059:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.087067:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.087086:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826347.087546:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dfcc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3652:x1874663346300800:12345-0@lo:103:ldlm_bl.0 Request processed in 1443us (1723us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826347.087555:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22031 00000100:00100000:0.0:1787826347.088075:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346300800:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826347.088330:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4bc91 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4bc8a expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826347.088347:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346300672:0@lo:104: 00010000:00010000:1.0:1787826347.088363:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.088372:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.088377:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826347.088448:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x198:0x0]: returning size 4096 00000004:00200000:1.0:1787826347.088453:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826347.088503:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bcde lrc: 0/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080195e expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.088520:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080195e expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.088529:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f794385a300) 00010000:00010000:1.0:1787826347.088532:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826347.088535:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:1.0:1787826347.088676:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226034:x1874663332657152:12345-192.168.203.56@tcp:101:bash.0 Request processed in 4324us (4669us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826347.088685:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68606 00000100:00100000:1.0:1787826347.095045:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826347.095265:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332657408 00000100:00100000:3.0:1787826347.095275:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332657408 00000100:00100000:3.0:1787826347.095380:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68607 00000100:00100000:3.0:1787826347.095393:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d00aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226034:x1874663332657408:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826347.095424:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826347.095427:0:35093:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826347.095456:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bcd7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801957 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.095475:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826347.095481:0:35093:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826347.095496:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4bcec lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801965 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.095632:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.095645:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.095655:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.095681:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.095688:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.095694:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826347.095826:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346300928:0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826347.095960:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826347.095996:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78474d7780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826347.096322:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346300928 00000100:00100000:2.0:1787826347.096328:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346300928 00000100:00100000:2.0:1787826347.096372:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23997 00000100:00100000:2.0:1787826347.096381:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35093:x1874663346300928:12345-0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826347.096464:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35093:x1874663346300928:12345-0@lo:1000:mdt00_010.0 Request processed in 83us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826347.096472:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23997 00000100:00100000:3.0:1787826347.096619:0:35093:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957455 gen 2 00000004:00080000:3.0:1787826347.096625:0:35093:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957455 committed transno 8589957455 00000100:00100000:3.0:1787826347.096655:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346300928:0@lo:1000:mdt00_010.0 00000004:00080000:3.0:1787826347.097170:0:35093:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4cd:0x0], index 1: 0 00010000:00010000:3.0:1787826347.097323:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bd01 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097338:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bd01 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097348:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bd01 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097385:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097392:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097409:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097417:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097423:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826347.097428:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097435:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097446:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826347.097449:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826347.097453:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097460:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bcf3 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826347.097475:0:35093:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000027a9c1dc reply state 0000000054d16e4c transno 4294985458 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097485:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097491:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f79785cc000) 00010000:00010000:3.0:1787826347.097493:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826347.097495:0:35093:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826347.097527:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4bcec lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801965 expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097541:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bd01 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801965 expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.097549:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:3.0:1787826347.097551:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826347.097553:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826347.097626:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d00aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226034:x1874663332657408:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2234us (2591us total) trans 4294985458 rc 0/0 00000100:00100000:3.0:1787826347.097633:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68607 00000100:00100000:3.0:1787826347.104561:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826347.104816:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332657536 00000100:00100000:0.0:1787826347.104827:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332657536 00000100:00080000:1.0:1787826347.109258:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826347.112394:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30867 00000100:00100000:1.0:1787826347.112425:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796eda7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226034:x1874663332657536:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826347.112445:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826347.112469:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080196c expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826347.112499:0:17826:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4cd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787826347.112524:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=8159s v=(0 0) 00010000:00010000:1.0:1787826347.112530:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826347.112541:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 8 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.112560:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 8 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826347.112584:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826347.112720:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796eda7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226034:x1874663332657536:12345-192.168.203.56@tcp:101:bash.0 Request processed in 303us (8169us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826347.112728:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30867 00000100:00100000:2.0:1787826347.124002:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826347.126374:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332657792 00000100:00100000:0.0:1787826347.126390:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332657792 00000100:00100000:0.0:1787826347.126470:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9380 00000100:00100000:0.0:1787826347.126482:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226034:x1874663332657792:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826347.126836:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:226034:x1874663332657792:12345-192.168.203.56@tcp:35:bash.0 Request processed in 354us (2843us total) trans 4294985459 rc 0/0 00000100:00100000:0.0:1787826347.126843:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9380 00000100:00100000:1.0:1787826347.131173:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826347.131271:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332658048 00000100:00100000:2.0:1787826347.131282:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332658048 00000100:00100000:2.0:1787826347.131345:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68608 00000100:00100000:2.0:1787826347.131356:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226034:x1874663332658048:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826347.131388:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826347.131391:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826347.131430:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4bce5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080195e expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.131451:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f794385a300) 00010000:00010000:2.0:1787826347.131461:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826347.131477:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd16 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801973 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.131631:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.131644:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.131653:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.131664:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.131811:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.131818:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826347.132417:0:17813:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x522:0x0], index 0: 0 00010000:00010000:2.0:1787826347.132563:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bd2b lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.132587:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bd2b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.132596:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bd2b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.132629:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.132637:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.132643:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.132650:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.132656:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826347.132661:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.134235:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.134254:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f7961f9ab00) 00010000:00010000:2.0:1787826347.134257:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826347.134263:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 3/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.134270:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd1d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826347.134289:0:17813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000093895a3 reply state 000000009cb8efff transno 8589957456 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.134301:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.134308:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x4ec94fd6 (ffff9f7961f9a200) 00010000:00010000:2.0:1787826347.134310:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826347.134314:0:17813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826347.134365:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd16 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801973 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.134410:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bd2b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801973 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826347.134418:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f7961f9ae00) 00010000:00010000:2.0:1787826347.134420:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826347.134423:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826347.134506:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226034:x1874663332658048:12345-192.168.203.56@tcp:101:bash.0 Request processed in 3151us (3343us total) trans 8589957456 rc 0/0 00000100:00100000:2.0:1787826347.134515:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68608 00000100:00100000:2.0:1787826347.141543:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826347.141632:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332658176 00000100:00100000:0.0:1787826347.141642:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332658176 00000100:00100000:0.0:1787826347.141713:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30868 00000100:00100000:0.0:1787826347.141723:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226034:x1874663332658176:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826347.141739:0:35352:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826347.141758:0:35352:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080197a expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826347.141784:0:35352:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x522:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787826347.141802:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x522:0x0].0x0 since=8159s v=(0 0) 00010000:00010000:0.0:1787826347.141806:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x522:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826347.141814:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 9 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.141830:0:35352:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 9 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.141857:0:35352:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826347.141906:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226034:x1874663332658176:12345-192.168.203.56@tcp:101:bash.0 Request processed in 185us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826347.141913:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30868 00000100:00080000:1.0:1787826347.142217:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826347.145156:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826347.145257:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332658432 00000100:00100000:2.0:1787826347.145267:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332658432 00000100:00100000:2.0:1787826347.145318:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9381 00000100:00100000:2.0:1787826347.145327:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226034:x1874663332658432:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:2.0:1787826347.145652:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226034:x1874663332658432:12345-192.168.203.56@tcp:35:bash.0 Request processed in 325us (546us total) trans 8589957457 rc 0/0 00000100:00100000:2.0:1787826347.145661:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9381 00000100:00100000:1.0:1787826347.168397:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826347.168944:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332658560 00000100:00100000:0.0:1787826347.168954:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332658560 00000100:00100000:0.0:1787826347.169023:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11177 00000100:00100000:0.0:1787826347.169034:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:15672:x1874663332658560:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826347.169050:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826347.169077:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4bd2b lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801973 expref: 12 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.169095:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f7961f9ae00) 00010000:00010000:0.0:1787826347.169155:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bc9f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801942 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.169166:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826347.169174:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bcbb lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801949 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826347.169183:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f794385ae00) 00010000:00010000:0.0:1787826347.169189:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826347.169250:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:15672:x1874663332658560:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 218us (864us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826347.169262:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11177 00000100:00100000:2.0:1787826347.174168:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826347.174766:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332658688 00000100:00100000:3.0:1787826347.174777:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332658688 00000100:00100000:3.0:1787826347.174850:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11178 00000100:00100000:3.0:1787826347.174861:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:15672:x1874663332658688:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826347.174878:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826347.174904:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bb1e lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dbf:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018f5 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.174933:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dbf:0x0].0x0 (ffff9f7843326100) 00010000:00010000:3.0:1787826347.174945:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826347.175008:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:15672:x1874663332658688:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 149us (849us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826347.175016:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11178 00000100:00100000:1.0:1787826347.177906:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826347.178051:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332658816 00000100:00100000:3.0:1787826347.178062:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332658816 00000100:00100000:3.0:1787826347.178163:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11179 00000100:00100000:3.0:1787826347.178176:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:34880:x1874663332658816:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826347.178191:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826347.178219:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bd01 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801965 expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.178241:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:3.0:1787826347.178258:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69000/0xe21309aa81f4bcc9 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801950 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.178268:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826347.178277:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4ba84 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e56:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608018d2 expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826347.178286:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e56:0x0].0x0 (ffff9f794385a400) 00010000:00010000:3.0:1787826347.178292:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826347.178345:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:34880:x1874663332658816:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 173us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826347.178352:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11179 00010000:00010000:0.0:1787826347.182007:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826348.215444:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826348.531189:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bbc6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826348.531213:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bbc6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.531266:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 2/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbc6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.531274:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826348.531301:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69a00/0xe21309aa81f4bbbf lrc: 0/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bbc6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826348.531322:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985459 is committed 00000020:00080000:2.0:1787826348.531339:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784d067000 refcount 2 committed 0 result 0 batchid 3437 00000020:00080000:2.0:1787826348.531344:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc2c400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d067180 00000020:00080000:2.0:1787826348.531348:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1541 00000020:00080000:2.0:1787826348.531351:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc2c100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7849237c00 00000020:00080000:2.0:1787826348.531354:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1223 00000001:00080000:2.0:1787826348.531363:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985456 is committed 00000020:00080000:2.0:1787826348.531367:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7977cfc600 refcount 2 committed 0 result 0 batchid 3436 00000020:00080000:2.0:1787826348.531370:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc2cb00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7850703a80 00000020:00080000:2.0:1787826348.531372:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1540 00000020:00080000:2.0:1787826348.531374:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc2c000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7851fe3800 00000020:00080000:2.0:1787826348.531377:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1221 00000004:00080000:2.0:1787826348.531386:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 368 committed 00000020:00080000:2.0:1787826348.531414:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784a358780 refcount 2 committed 0 result 0 batchid 3435 00000020:00080000:2.0:1787826348.531417:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1b100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a358c00 00000020:00080000:2.0:1787826348.531420:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1539 00000020:00080000:2.0:1787826348.531421:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cd1b600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7845ee1400 00000020:00080000:2.0:1787826348.531424:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1220 00000020:00080000:2.0:1787826348.531432:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7845d45480 refcount 2 committed 0 result 0 batchid 3434 00000020:00080000:2.0:1787826348.531435:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed6a000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7845d45900 00000020:00080000:2.0:1787826348.531437:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1538 00000020:00080000:2.0:1787826348.531439:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed6a700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a5f3a00 00000020:00080000:2.0:1787826348.531441:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1219 00000001:00080000:2.0:1787826348.531449:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985450 is committed 00000020:00080000:2.0:1787826348.531463:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3433 batchid updated 00000100:00100000:1.0:1787826348.534624:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b306a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346301056:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826348.534708:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826348.534821:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346301056 00000100:00100000:1.0:1787826348.534828:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346301056 00000100:00100000:1.0:1787826348.534885:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11180 00000100:00100000:1.0:1787826348.534895:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3654:x1874663346301056:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826348.534912:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826348.534937:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4bbc6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x197:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bbbf expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.534956:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x197:0x0].0x5a734ce9 (ffff9f7843326d00) 00010000:00010000:1.0:1787826348.534967:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826348.535003:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346301056:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 109us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826348.535011:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11180 00000100:00100000:1.0:1787826348.535144:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b306a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346301056:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826348.535161:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826348.535166:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346301184:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826348.535195:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826348.535221:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346301184 00000100:00100000:1.0:1787826348.535225:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346301184 00000100:00100000:1.0:1787826348.535260:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30869 00000100:00100000:1.0:1787826348.535267:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346301184:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1787826348.535288:0:8443:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1231 count 1 00010000:00010000:1.0:1787826348.535312:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.535331:0:8443:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 8 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.535340:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4cf:0x0].0x0 since=7s v=(0 0) 00010000:00010000:1.0:1787826348.535345:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826348.535349:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.535375:0:8443:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 8 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.535387:0:8443:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 8 pid: 35096 timeout: 8260 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826348.535417:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346301312:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826348.535480:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ae40 going to sleep for 11 seconds 00000100:00080000:1.0:1787826348.535492:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787826348.535978:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.535988:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.535996:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536002:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826348.536007:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536014:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536026:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826348.536029:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826348.536032:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536038:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bc7c lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536056:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536062:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536068:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536072:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826348.536075:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536080:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536086:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826348.536088:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826348.536090:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536095:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bc83 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536137:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536144:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536149:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536154:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826348.536157:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536167:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536174:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f79785cc000) 00010000:00010000:3.0:1787826348.536176:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826348.536179:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826348.536184:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bcfa lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787826348.536209:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3437 committed batchid 3433 00000004:00080000:3.0:1787826348.536319:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d067a80 with version 6872 00000020:00080000:3.0:1787826348.536331:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3430 cancel update log [0x200000401:0x3:0x0].1534: rc = 0 00000020:00080000:3.0:1787826348.536375:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3430 cancel update log [0x240000400:0x3:0x0].1215: rc = 0 00000004:00080000:3.0:1787826348.536413:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d0673c0 with version 6873 00000020:00080000:3.0:1787826348.536416:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3431 cancel update log [0x200000401:0x3:0x0].1535: rc = 0 00000020:00080000:3.0:1787826348.536437:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3431 cancel update log [0x240000400:0x3:0x0].1216: rc = 0 00000004:00080000:3.0:1787826348.536460:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d82ec00 with version 6874 00000020:00080000:3.0:1787826348.536463:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3432 cancel update log [0x200000401:0x3:0x0].1536: rc = 0 00000020:00080000:3.0:1787826348.536482:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3432 cancel update log [0x240000400:0x3:0x0].1217: rc = 0 00000004:00080000:3.0:1787826348.536502:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f79438e2540 with version 6875 00000020:00080000:3.0:1787826348.536505:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3433 cancel update log [0x200000401:0x3:0x0].1537: rc = 0 00000020:00080000:3.0:1787826348.536523:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3433 cancel update log [0x240000400:0x3:0x0].1218: rc = 0 00000020:00080000:3.0:1787826348.536530:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3433 00000004:00080000:2.0:1787826348.536603:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6872 rpc_version 6872 00000020:00100000:2.0:1787826348.536627:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826348.536636:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b304000 x1874663346301568/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826348.536669:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346301568:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826348.536717:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826348.536744:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826348.536890:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346301568 00000100:00100000:1.0:1787826348.536898:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346301568 00000100:00100000:1.0:1787826348.537013:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23998 00000100:00100000:1.0:1787826348.537022:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346301568:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826348.537174:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346301568:12345-0@lo:1000:osp_up0-1.0 Request processed in 154us (456us total) trans 8589957458 rc 0/0 00000100:00100000:1.0:1787826348.537181:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23998 00000100:00100000:3.0:1787826348.537260:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346301568:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826348.537271:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6873 rpc_version 6873 00000020:00100000:3.0:1787826348.537284:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826348.537288:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784a91ce00 x1874663346301824/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826348.537302:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91ce00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346301824:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826348.537331:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826348.551131:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826348.558606:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346301824 00000100:00100000:2.0:1787826348.558623:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346301824 00000100:00100000:2.0:1787826348.558715:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23999 00000100:00100000:2.0:1787826348.558725:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346301824:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826348.558948:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346301824:12345-0@lo:1000:osp_up0-1.0 Request processed in 224us (21614us total) trans 8589957459 rc 0/0 00000100:00100000:2.0:1787826348.558956:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23999 00000100:00100000:2.0:1787826348.559028:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91ce00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346301824:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826348.559044:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6874 rpc_version 6874 00000020:00100000:2.0:1787826348.559067:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826348.559073:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b305500 x1874663346302080/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826348.559095:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346302080:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826348.559156:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826348.559180:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826348.559194:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346302080 00000100:00100000:2.0:1787826348.559198:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346302080 00000100:00100000:2.0:1787826348.559216:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24000 00000100:00100000:2.0:1787826348.559222:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346302080:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826348.559296:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346302080:12345-0@lo:1000:osp_up0-1.0 Request processed in 74us (140us total) trans 8589957460 rc 0/0 00000100:00100000:2.0:1787826348.559303:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24000 00000100:00100000:2.0:1787826348.559334:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346302080:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826348.559341:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6875 rpc_version 6875 00000020:00100000:2.0:1787826348.559352:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826348.559357:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b304a80 x1874663346302336/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826348.559368:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346302336:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826348.559408:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826348.559423:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826348.559433:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346302336 00000100:00100000:2.0:1787826348.559437:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346302336 00000100:00100000:2.0:1787826348.559454:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24001 00000100:00100000:2.0:1787826348.559460:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346302336:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826348.559527:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346302336:12345-0@lo:1000:osp_up0-1.0 Request processed in 66us (118us total) trans 8589957461 rc 0/0 00000100:00100000:2.0:1787826348.559533:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24001 00000100:00100000:2.0:1787826348.559561:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346302336:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826348.563890:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826348.563959:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332658944 00000100:00100000:1.0:1787826348.563968:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332658944 00010000:00010000:1.0:1787826348.564011:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 9 pid: 35096 timeout: 8260 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826348.564045:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22032 00000100:00100000:1.0:1787826348.564055:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332658944:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826348.564068:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826348.564074:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 9 pid: 35096 timeout: 8260 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.564088:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 9 pid: 35096 timeout: 8260 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.564149:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826348.564155:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4cf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826348.564161:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.564172:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826348.564196:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826348.564247:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332658944:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 191us (365us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826348.564253:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22032 00010000:00010000:2.0:1787826348.564461:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cef0a631/0xe21309aa81f4b9dc lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801893 expref: 8 pid: 35096 timeout: 8260 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826348.564478:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b660a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346301312:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826348.564501:0:8443:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564512:0:8443:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564525:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564535:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564543:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564561:0:8443:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826348.564734:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346301184:12345-0@lo:6:osp-syn-0-1.0 Request processed in 29466us (29537us total) trans 4294973858 rc 0/0 00000100:00100000:2.0:1787826348.564743:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30869 00010000:00010000:2.0:1787826348.564763:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826348.564769:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564777:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826348.564781:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564789:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564802:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564810:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b1181e5/0xe21309aa81f4bd40 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4cf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826348.564834:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826348.564836:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826348.564838:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787826348.571458:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7851240000/2, rc 0, transno 4294973858 00000100:00100000:1.0:1787826348.571473:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346301184:0@lo:6:osp-syn-0-1.0 00000001:00080000:2.0:1787826348.572012:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976167 is committed 00000001:00080000:2.0:1787826348.572025:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976166 is committed 00000001:00080000:2.0:1787826348.572035:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976165 is committed 00010000:00010000:1.0:1787826348.974858:0:167027:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:1.0:1787826348.974914:0:167027:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:2.0:1787826349.233799:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826350.253556:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826350.303590:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826350.303768:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332659200 00000100:00100000:3.0:1787826350.303778:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332659200 00000100:00100000:3.0:1787826350.303849:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14354 00000100:00100000:3.0:1787826350.303860:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332659200:12345-192.168.203.56@tcp:400:kworker.0 00000100:00080000:2.0:1787826350.307878:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826350.312506:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332659200:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 8635us (8913us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.312522:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14354 00000100:00100000:1.0:1787826350.312562:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826350.312598:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332659456 00000100:00100000:1.0:1787826350.312607:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332659456 00000100:00100000:3.0:1787826350.312694:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826350.312717:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826350.312743:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826350.312764:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826350.312784:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826350.312802:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826350.312826:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826350.312846:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332659712 00000100:00100000:3.0:1787826350.312850:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332659712 00000100:00100000:3.0:1787826350.312875:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332659328 00000100:00100000:1.0:1787826350.312876:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332659968 00000100:00100000:1.0:1787826350.312879:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332659968 00000100:00100000:3.0:1787826350.312879:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332659328 00000100:00100000:3.0:1787826350.312897:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332659840 00000100:00100000:3.0:1787826350.312900:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332659840 00000100:00100000:3.0:1787826350.312921:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14355 00000100:00100000:3.0:1787826350.312929:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332659712:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826350.312998:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332659712:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 69us (303us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.313006:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14355 00000100:00100000:3.0:1787826350.313019:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14356 00000100:00100000:3.0:1787826350.313025:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332659328:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826350.313055:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332659328:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.313061:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14356 00000100:00100000:3.0:1787826350.313072:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14357 00000100:00100000:3.0:1787826350.313078:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332659840:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826350.313101:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332659840:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (299us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.313135:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14357 00000100:00100000:3.0:1787826350.313592:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332659072 00000100:00100000:3.0:1787826350.313599:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332659072 00000100:00100000:3.0:1787826350.313630:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2236 00000100:00100000:3.0:1787826350.313638:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79747db450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332659072:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826350.313719:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332659584 00000100:00100000:1.0:1787826350.313724:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332659584 00000100:00100000:3.0:1787826350.313724:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79747db450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332659072:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 87us (981us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.313731:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2236 00000100:00100000:1.0:1787826350.313741:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332660096 00000100:00100000:1.0:1787826350.313743:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332660096 00000100:00100000:1.0:1787826350.313765:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8863 00000100:00100000:1.0:1787826350.313786:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332659456:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826350.313815:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332659456:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 44us (1253us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.319869:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8864 00000100:00100000:3.0:1787826350.319893:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332659968:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826350.319972:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332659968:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 90us (7253us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.319979:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8864 00000100:00100000:3.0:1787826350.320014:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8865 00000100:00100000:3.0:1787826350.320021:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332659584:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826350.320044:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332659584:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (7260us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.320049:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8865 00000100:00100000:3.0:1787826350.320061:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8866 00000100:00100000:3.0:1787826350.320067:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332660096:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826350.320085:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332660096:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 19us (7260us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826350.320091:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8866 00000100:00100000:1.0:1787826350.320909:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8863 00000100:00080000:1.0:1787826350.320939:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826350.320946:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826350.320949:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787826350.320951:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826351.084816:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826351.084892:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826351.084918:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346302592:0@lo:400:kworker.0 00000100:00100000:3.0:1787826351.085001:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826351.091527:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346302592 00000100:00100000:1.0:1787826351.091544:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346302592 00000100:00100000:1.0:1787826351.091619:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24002 00000100:00100000:1.0:1787826351.091629:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346302592:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826351.091747:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346302592:12345-0@lo:400:kworker.0 Request processed in 119us (6746us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826351.091755:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24002 00000100:00100000:3.0:1787826351.092681:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346302592:0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.095578:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346302464:0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.095669:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826351.095867:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346302464 00000100:00100000:1.0:1787826351.095876:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346302464 00000100:00100000:1.0:1787826351.095933:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2237 00000100:00100000:1.0:1787826351.095944:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79747d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346302464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826351.096008:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79747d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346302464:12345-0@lo:400:kworker.0 Request processed in 66us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826351.096016:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2237 00000100:00100000:3.0:1787826351.096324:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346302464:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.098642:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346302720:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.098713:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826351.098746:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346303232:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.098781:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826351.098799:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346303744:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.098821:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826351.098847:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1787826351.098852:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346302848:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.098873:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826351.098986:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346302976:0@lo:400:kworker.0 00000100:00100000:3.0:1787826351.099074:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e763800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346303104:0@lo:400:kworker.0 00000100:00100000:1.0:1787826351.099188:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346303744 00000100:00100000:1.0:1787826351.099196:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346303744 00000100:00100000:1.0:1787826351.099756:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8867 00000100:00100000:1.0:1787826351.099768:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346303744:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.105195:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826351.105259:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346303488:0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.105307:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826351.105330:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346304000:0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.105356:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826351.105425:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346303488 00000100:00100000:2.0:1787826351.105433:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346303488 00000100:00100000:2.0:1787826351.107322:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8868 00000100:00100000:2.0:1787826351.107339:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346303488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.107446:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346303488:12345-0@lo:400:kworker.0 Request processed in 113us (2137us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826351.108250:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8868 00000100:00100000:2.0:1787826351.109527:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976165 gen 2 00000004:00080000:2.0:1787826351.109536:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784b75ad80, transno 4294976165 00000100:00100000:2.0:1787826351.109560:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346303488:0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.109630:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346302720 00000100:00100000:2.0:1787826351.109637:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346302720 00000100:00100000:2.0:1787826351.109682:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346303232 00000100:00100000:2.0:1787826351.109685:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346303232 00000100:00100000:2.0:1787826351.109705:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346302848 00000100:00100000:2.0:1787826351.109708:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346302848 00000100:00100000:2.0:1787826351.109731:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346302976 00000100:00100000:2.0:1787826351.109733:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346302976 00000100:00100000:2.0:1787826351.109748:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346304000 00000100:00100000:2.0:1787826351.109750:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346304000 00000100:00100000:2.0:1787826351.109777:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14358 00000100:00100000:2.0:1787826351.109785:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346302720:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.109859:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346302720:12345-0@lo:400:kworker.0 Request processed in 74us (11146us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826351.109865:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14358 00000100:00100000:2.0:1787826351.109879:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14359 00000100:00100000:2.0:1787826351.109985:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346303232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.110015:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346303232:12345-0@lo:400:kworker.0 Request processed in 131us (11234us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826351.110311:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14359 00000100:00100000:2.0:1787826351.110330:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14360 00000100:00100000:2.0:1787826351.110338:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346302848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.110381:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346302848:12345-0@lo:400:kworker.0 Request processed in 35us (11497us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826351.110387:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14360 00000100:00100000:2.0:1787826351.110399:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14361 00000100:00100000:2.0:1787826351.110405:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346302976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.110436:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346302976:12345-0@lo:400:kworker.0 Request processed in 32us (5250us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826351.110441:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14361 00000100:00100000:2.0:1787826351.110452:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14362 00000100:00100000:2.0:1787826351.110458:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346304000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.110484:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346304000:12345-0@lo:400:kworker.0 Request processed in 26us (5128us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826351.110489:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14362 00000100:00100000:2.0:1787826351.110534:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346302976:0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.110546:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346304000:0@lo:400:kworker.0 00000100:00080000:2.0:1787826351.110630:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826351.110637:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826351.110639:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826351.110643:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826351.111003:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346302848:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.111015:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346303360:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.111055:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826351.111077:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346303872:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.111102:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826351.111167:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346302720:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.111178:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346303232:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.111324:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346303872 00000100:00100000:0.0:1787826351.111329:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346303872 00000100:00100000:0.0:1787826351.111381:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8869 00000100:00100000:0.0:1787826351.111390:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346303872:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.111433:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346303872:12345-0@lo:400:kworker.0 Request processed in 45us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826351.111439:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8869 00000100:00080000:0.0:1787826351.111839:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826351.111860:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976167 gen 2 00000004:00080000:0.0:1787826351.111865:0:3654:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784d9e9500, transno 4294976167 00000100:00100000:0.0:1787826351.111928:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346303872:0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.112040:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346303360 00000100:00100000:0.0:1787826351.112045:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346303360 00000100:00100000:0.0:1787826351.112082:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14363 00000100:00100000:0.0:1787826351.112089:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346303360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826351.118740:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346303360:12345-0@lo:400:kworker.0 Request processed in 6641us (7674us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826351.118757:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14363 00000100:00100000:3.0:1787826351.119406:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826351.119451:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346303616:0@lo:400:kworker.0 00000100:00100000:3.0:1787826351.119507:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826351.126771:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346303104 00000100:00100000:1.0:1787826351.126788:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346303104 00000100:00100000:1.0:1787826351.126959:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8870 00000100:00100000:1.0:1787826351.126971:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346303104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826351.127069:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346303104:12345-0@lo:400:kworker.0 Request processed in 101us (7666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826351.127076:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8870 00000100:00100000:1.0:1787826351.127123:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346303616 00000100:00100000:1.0:1787826351.127128:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346303616 00000100:00100000:1.0:1787826351.127170:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24003 00000100:00100000:1.0:1787826351.127177:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346303616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826351.129270:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346303616:12345-0@lo:400:kworker.0 Request processed in 2085us (9754us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826351.129284:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24003 00000100:00100000:1.0:1787826351.129366:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346303744:12345-0@lo:400:kworker.0 Request processed in 29601us (30542us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826351.129384:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8867 00000100:00100000:2.0:1787826351.129621:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346303360:0@lo:400:kworker.0 00000100:00100000:2.0:1787826351.129637:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346303744:0@lo:400:kworker.0 00000100:00080000:0.0:1787826351.131962:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787826351.131973:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826351.136437:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e763800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346303104:0@lo:400:kworker.0 00000100:00100000:3.0:1787826351.136456:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985450 gen 2 00000004:00080000:3.0:1787826351.136462:0:3653:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985449 committed transno 4294985450 00000020:00080000:3.0:1787826351.136468:0:3653:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f78434e4b40 refcount 2 committed 0 result 0 batchid 4294979087 00000020:00080000:3.0:1787826351.136472:0:3653:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc2c300 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78434e4d80 00000020:00080000:3.0:1787826351.136476:0:3653:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].944 00000020:00080000:3.0:1787826351.136479:0:3653:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc2cc00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e774a00 00000020:00080000:3.0:1787826351.136482:0:3653:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].941 00000004:00080000:3.0:1787826351.136518:0:3653:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985450 committed transno 4294985450 00000100:00100000:3.0:1787826351.136532:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346303616:0@lo:400:kworker.0 00000020:00080000:0.0:1787826351.137559:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979087 committed batchid 4294979086 00000020:00080000:0.0:1787826351.137651:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979086 00010000:00010000:1.0:1787826351.276614:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826351.327981:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826351.328024:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826351.328144:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332660352 00000100:00100000:0.0:1787826351.328162:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332660352 00000100:00100000:0.0:1787826351.328217:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30870 00000100:00100000:0.0:1787826351.328226:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332660352:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:0.0:1787826351.328322:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332660352:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 97us (351us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826351.328329:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30870 00000100:00100000:2.0:1787826351.329073:0:24158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332660224 00000100:00100000:2.0:1787826351.329080:0:24158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332660224 00002000:00100000:2.0:1787826351.329138:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000400:0x4cd:0x0] (0->5) req@ffff9f784d9e9500 x1874663332660224/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:707/0 lens 496/0 e 0 to 0 dl 1787826362 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826351.329157:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d9e9500 with x1874663332660224 ext(0->5) 00002000:00010000:2.0:1787826351.329163:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784d9e9500 00000100:00100000:2.0:1787826351.329184:0:24158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27661 00000100:00100000:2.0:1787826351.329192:0:24158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332660224:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:2.0:1787826351.332646:0:24158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332660224:12345-192.168.203.56@tcp:4:bash.0 Request processed in 3443us (4609us total) trans 4294976168 rc 0/0 00000100:00100000:2.0:1787826351.332660:0:24158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27661 00002000:00100000:2.0:1787826351.332672:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x2c0000400:0x4cd:0x0] (0->5) req@ffff9f784d9e9500 x1874663332660224/t4294976168(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:707/0 lens 496/448 e 0 to 0 dl 1787826362 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826351.332682:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d9e9500 with x1874663332660224 ext(0->5) 00002000:00010000:2.0:1787826351.332688:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784d9e9500 00000100:00100000:2.0:1787826351.344015:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826351.348924:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826351.349059:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332660608 00000100:00100000:0.0:1787826351.349069:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332660608 00000100:00100000:0.0:1787826351.349156:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30871 00000100:00100000:0.0:1787826351.349167:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332660608:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:0.0:1787826351.349273:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332660608:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 108us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826351.349281:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30871 00000100:00100000:1.0:1787826351.350765:0:24220:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332660480 00000100:00100000:1.0:1787826351.350776:0:24220:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332660480 00002000:00100000:0.0:1787826351.351312:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000401:0x522:0x0] (0->5) req@ffff9f784b306a00 x1874663332660480/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:707/0 lens 496/0 e 0 to 0 dl 1787826362 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826351.351325:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b306a00 with x1874663332660480 ext(0->5) 00002000:00010000:0.0:1787826351.351435:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784b306a00 00000100:00100000:0.0:1787826351.351514:0:24220:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27662 00000100:00100000:0.0:1787826351.351525:0:24220:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332660480:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:1.0:1787826351.353711:0:24220:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2393:x1874663332660480:12345-192.168.203.56@tcp:4:bash.0 Request processed in 2178us (9695us total) trans 4294973859 rc 0/0 00000100:00100000:1.0:1787826351.353728:0:24220:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27662 00002000:00100000:1.0:1787826351.353741:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_012: refresh rw locks for [0x280000401:0x522:0x0] (0->5) req@ffff9f784b306a00 x1874663332660480/t4294973859(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:707/0 lens 496/448 e 0 to 0 dl 1787826362 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826351.353751:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b306a00 with x1874663332660480 ext(0->5) 00002000:00010000:1.0:1787826351.353757:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784b306a00 00000001:00080000:1.0:1787826351.365160:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973859 is committed 00000001:00080000:1.0:1787826351.365179:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973858 is committed 00000100:00100000:0.0:1787826351.597252:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346304128:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826351.597334:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826351.597407:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346304128 00000100:00100000:0.0:1787826351.597414:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346304128 00000100:00100000:0.0:1787826351.597475:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6255 00000100:00100000:0.0:1787826351.597485:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346304128:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826351.597508:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826351.597556:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346304128:12345-0@lo:13:osp-pre-1-1.0 Request processed in 73us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826351.597564:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6255 00000100:00100000:0.0:1787826351.597613:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346304128:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826352.109234:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787826352.109260:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346304256:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826352.109337:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.109396:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346304256 00000100:00100000:0.0:1787826352.109404:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346304256 00000100:00100000:0.0:1787826352.109464:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24004 00000100:00100000:0.0:1787826352.109474:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346304256:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787826352.109509:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442758656 avail 1296519168 00000100:00100000:0.0:1787826352.109554:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346304256:12345-0@lo:41:osp-pre-0-1.0 Request processed in 82us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.109561:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24004 00000100:00100000:0.0:1787826352.109608:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346304256:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826352.110024:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346304384:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826352.110055:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.110088:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346304384 00000100:00100000:0.0:1787826352.110092:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346304384 00000100:00100000:0.0:1787826352.110142:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24005 00000100:00100000:0.0:1787826352.110150:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346304384:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1787826352.110171:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443381248 avail 1297141760 00000100:00100000:0.0:1787826352.110207:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346304384:12345-0@lo:41:osp-pre-1-0.0 Request processed in 59us (152us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.110214:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24005 00000100:00100000:0.0:1787826352.110249:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346304384:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826352.111344:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346304512:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826352.111409:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826352.126205:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826352.126227:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967910000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346304640:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826352.126303:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.129229:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346304512 00000100:00100000:0.0:1787826352.129242:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346304512 00000100:00100000:0.0:1787826352.129297:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346304640 00000100:00100000:0.0:1787826352.129300:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346304640 00000100:00100000:0.0:1787826352.129323:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6256 00000100:00100000:0.0:1787826352.129332:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346304512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787826352.129357:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826352.129476:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346304512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 145us (18066us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.129483:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6256 00000100:00100000:0.0:1787826352.129513:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6257 00000100:00100000:0.0:1787826352.129520:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346304640:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787826352.129531:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662401536 00000100:00100000:0.0:1787826352.129558:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346304640:12345-0@lo:13:osp-pre-0-1.0 Request processed in 39us (3255us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.129563:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6257 00000100:00100000:0.0:1787826352.129589:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346304768:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826352.129626:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.129665:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346304768 00000100:00100000:0.0:1787826352.129668:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346304768 00000100:00100000:0.0:1787826352.129685:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6258 00000100:00100000:0.0:1787826352.129690:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346304768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787826352.129699:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662401536 00000100:00100000:0.0:1787826352.129727:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346304768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 36us (101us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.129733:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6258 00000100:00100000:0.0:1787826352.129980:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845affb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346304768:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826352.130027:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973858 gen 2 00000004:00080000:0.0:1787826352.130033:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7851240000, transno 4294973858 00000100:00100000:0.0:1787826352.130059:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967910000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346304640:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826352.130076:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346304512:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826352.282526:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826352.283377:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332660736 00000100:00100000:0.0:1787826352.283387:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332660736 00000100:00100000:0.0:1787826352.283465:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68609 00000100:00100000:0.0:1787826352.283477:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226215:x1874663332660736:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826352.283499:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826352.283522:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bd47 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801981 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.283588:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.283601:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.283615:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826352.283666:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826352.283670:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826352.283691:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.283717:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4bd47 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.283733:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.283748:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826352.283751:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826352.283763:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826352.283834:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226215:x1874663332660736:12345-192.168.203.56@tcp:101:cat.0 Request processed in 359us (1317us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.283841:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68609 00000100:00080000:1.0:1787826352.284798:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826352.291417:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826352.291719:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332660864 00000100:00100000:3.0:1787826352.291729:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332660864 00000100:00100000:3.0:1787826352.291810:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68610 00000100:00100000:3.0:1787826352.291821:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845affb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226215:x1874663332660864:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826352.291842:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826352.291866:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bd55 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801988 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.291936:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.291951:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.291963:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.291978:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.291985:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.291991:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292065:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4bd6a lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292074:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4bd6a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292080:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4bd6a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826352.292101:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4bd6a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292134:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292142:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292147:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292154:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292159:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826352.292164:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292170:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292182:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826352.292185:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826352.292189:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292194:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bd5c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292207:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292212:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292217:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292222:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826352.292225:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292230:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292237:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cdd00) 00010000:00010000:3.0:1787826352.292239:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826352.292241:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292247:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4bd63 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292277:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bd55 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801988 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292291:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4bd6a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801988 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.292298:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cd900) 00010000:00010000:3.0:1787826352.292300:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826352.292302:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826352.292376:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845affb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226215:x1874663332660864:12345-192.168.203.56@tcp:101:cat.0 Request processed in 557us (969us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826352.292384:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68610 00010000:00010000:0.0:1787826352.301632:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826352.304679:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826352.306372:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332660992 00000100:00100000:0.0:1787826352.306388:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332660992 00000100:00100000:0.0:1787826352.306484:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68611 00000100:00100000:0.0:1787826352.306508:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:226215:x1874663332660992:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826352.306530:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826352.306557:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4bd71 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080198f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.306622:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4bd78 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.306635:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4bd78 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.306649:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4bd78 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826352.306753:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e58:0x0]: returning size 4096 00000004:00200000:0.0:1787826352.306922:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826352.306970:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4bd71 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080198f expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.306991:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4bd78 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080198f expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.307001:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:0.0:1787826352.307004:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826352.307008:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826352.307084:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226215:x1874663332660992:12345-192.168.203.56@tcp:101:cat.0 Request processed in 590us (2414us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.307092:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68611 00000100:00080000:1.0:1787826352.307916:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826352.318814:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826352.318931:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332661120 00000100:00100000:2.0:1787826352.318941:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332661120 00000100:00100000:2.0:1787826352.319009:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68612 00000100:00100000:2.0:1787826352.319021:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226215:x1874663332661120:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826352.319040:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826352.319065:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bd7f lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801996 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.319192:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.319211:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.319222:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826352.319270:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19a:0x0]: returning size 4096 00000004:00200000:2.0:1787826352.319275:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826352.319314:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bd7f lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.319331:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.319339:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7952ca9200) 00010000:00010000:2.0:1787826352.319342:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826352.319345:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826352.319416:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226215:x1874663332661120:12345-192.168.203.56@tcp:101:cat.0 Request processed in 398us (613us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.319424:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68612 00000100:00100000:3.0:1787826352.328038:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826352.328164:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332661248 00000100:00100000:2.0:1787826352.328175:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332661248 00000100:00100000:2.0:1787826352.328236:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68613 00000100:00100000:2.0:1787826352.328248:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226215:x1874663332661248:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826352.328268:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826352.328292:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd8d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e876080199d expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.328380:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.328394:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.328404:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826352.328465:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x198:0x0]: returning size 4096 00000004:00200000:2.0:1787826352.328470:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826352.328777:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bd8d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.328796:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.328805:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:2.0:1787826352.328808:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826352.328812:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826352.328872:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226215:x1874663332661248:12345-192.168.203.56@tcp:101:cat.0 Request processed in 626us (845us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.328879:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68613 00000100:00100000:0.0:1787826352.337383:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826352.337878:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332661504 00000100:00100000:1.0:1787826352.337890:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332661504 00000100:00100000:3.0:1787826352.338049:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68614 00000100:00100000:3.0:1787826352.338063:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226215:x1874663332661504:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826352.338084:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826352.338120:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bd9b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019a4 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.338199:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.338213:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.338224:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.338240:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.338248:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.338255:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00020000:3.0:1787826352.338332:0:11629:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000020:00080000:1.0:1787826352.623309:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979087 batchid updated 00000001:00080000:1.0:1787826352.623343:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957461 is committed 00000001:00080000:1.0:1787826352.623388:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957457 is committed 00000020:00080000:0.0:1787826352.623447:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979087 00000004:00080000:0.0:1787826352.623529:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f78463186c0 with version 1836 00000020:00080000:0.0:1787826352.623550:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979087 cancel update log [0x240000404:0x2:0x0].944: rc = 0 00000020:00080000:0.0:1787826352.623632:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979087 cancel update log [0x200000bd0:0x2:0x0].941: rc = 0 00000020:00080000:0.0:1787826352.623643:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979087 00000004:00080000:1.0:1787826352.623690:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1836 rpc_version 1836 00000020:00100000:1.0:1787826352.623723:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826352.623734:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7967910000 x1874663346305024/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826352.623769:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967910000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346305024:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826352.623845:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826352.623886:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.624031:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346305024 00000100:00100000:0.0:1787826352.624039:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346305024 00000100:00100000:0.0:1787826352.624098:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24006 00000100:00100000:0.0:1787826352.624154:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:30909:x1874663346305024:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826352.624306:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346305024:12345-0@lo:1000:osp_up1-0.0 Request processed in 200us (460us total) trans 4294985460 rc 0/0 00000100:00100000:0.0:1787826352.624314:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24006 00010000:00010000:3.0:1787826352.624555:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.624566:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.624579:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.624585:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826352.624590:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.624598:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.624614:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x4ec94fd6 (ffff9f7961f9a200) 00010000:00010000:3.0:1787826352.624617:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826352.624622:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.624628:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4bd24 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826352.625471:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967910000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346305024:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826352.830256:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826352.830702:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332661632 00000100:00100000:2.0:1787826352.830713:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332661632 00000100:00100000:2.0:1787826352.830867:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68615 00000100:00100000:2.0:1787826352.830878:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226217:x1874663332661632:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826352.831734:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826352.831760:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bdb0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019ab expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.831828:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.831841:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.831854:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826352.831906:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826352.831910:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826352.831933:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.831959:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bdb0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.831978:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.831987:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826352.831990:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826352.831994:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826352.832301:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226217:x1874663332661632:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1424us (2053us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.832314:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68615 00000100:00080000:1.0:1787826352.834119:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826352.844162:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826352.844489:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332661760 00000100:00100000:3.0:1787826352.844500:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332661760 00000100:00100000:3.0:1787826352.844624:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68616 00000100:00100000:3.0:1787826352.844665:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226217:x1874663332661760:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826352.844688:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826352.844711:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bdbe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019b2 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844779:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844794:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844805:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844820:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844827:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844833:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844900:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844909:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844915:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826352.844937:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844951:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844957:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844967:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844973:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844978:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826352.844982:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844989:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.844999:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826352.845001:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826352.845005:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845010:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdc5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845024:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845029:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845034:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845039:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826352.845043:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845049:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845055:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cdc00) 00010000:00010000:3.0:1787826352.845057:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826352.845060:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845065:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdcc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845095:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bdbe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845152:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.845159:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cd900) 00010000:00010000:3.0:1787826352.845161:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826352.845164:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826352.845228:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226217:x1874663332661760:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 594us (1075us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826352.845752:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68616 00000100:00100000:0.0:1787826352.852983:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826352.853243:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332661888 00000100:00100000:2.0:1787826352.853254:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332661888 00000100:00100000:2.0:1787826352.853378:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68617 00000100:00100000:2.0:1787826352.853390:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:226217:x1874663332661888:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826352.853410:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826352.853431:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4bdda lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019b9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.853504:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4bde1 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.853518:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4bde1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.853531:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4bde1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826352.853628:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e58:0x0]: returning size 4096 00000004:00200000:2.0:1787826352.853633:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826352.853671:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4bdda lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019b9 expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.853689:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4bde1 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019b9 expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.853698:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:2.0:1787826352.853701:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826352.853705:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826352.853765:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226217:x1874663332661888:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 378us (792us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.853771:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68617 00000100:00080000:1.0:1787826352.855811:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826352.869838:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826352.873672:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332662016 00000100:00100000:3.0:1787826352.873688:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332662016 00000100:00100000:3.0:1787826352.873768:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68618 00000100:00100000:3.0:1787826352.873781:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226217:x1874663332662016:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826352.873803:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826352.873826:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bde8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019c0 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.873897:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 3/1,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.873912:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.873923:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.873938:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.873944:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.873950:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874039:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bdfd lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874047:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bdfd lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874054:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bdfd lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826352.874098:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dc0:0x0]: returning size 6 00000004:00200000:3.0:1787826352.874103:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826352.874173:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bdfd lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874186:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bdfd lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874195:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874200:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874206:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874213:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874219:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826352.874223:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874238:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874251:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:3.0:1787826352.874254:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826352.874258:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 3/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874263:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4bdef lrc: 0/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874279:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874284:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874289:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874293:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826352.874297:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874302:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874309:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x4ec94fd6 (ffff9f79785cdc00) 00010000:00010000:3.0:1787826352.874312:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826352.874314:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874320:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4bdf6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874350:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4bde8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019c0 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874375:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bdfd lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019c0 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.874388:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f79785cdb00) 00010000:00010000:3.0:1787826352.874390:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826352.874393:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826352.874460:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226217:x1874663332662016:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 682us (4633us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826352.874467:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68618 00000100:00100000:0.0:1787826352.884628:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826352.884775:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332662144 00000100:00100000:2.0:1787826352.884785:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332662144 00000100:00100000:2.0:1787826352.884846:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68619 00000100:00100000:2.0:1787826352.884857:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226217:x1874663332662144:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826352.884878:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826352.884901:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4be04 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019c7 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.884970:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 3/1,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.884984:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.884995:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885007:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885014:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885020:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885150:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4be19 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885160:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4be19 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885166:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4be19 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826352.885206:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e59:0x0]: returning size 6 00000004:00200000:2.0:1787826352.885210:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826352.885818:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4be19 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885835:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4be19 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885847:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885853:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885859:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885865:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885872:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826352.885876:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885885:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885898:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7952ca9200) 00010000:00010000:2.0:1787826352.885901:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826352.885905:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 3/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885912:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4be0b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885927:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885933:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885938:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885943:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826352.885946:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885951:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885959:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f79785cdd00) 00010000:00010000:2.0:1787826352.885961:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826352.885964:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.885970:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4be12 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.886002:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4be04 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019c7 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.886016:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4be19 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019c7 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.886023:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f7967ac8a00) 00010000:00010000:2.0:1787826352.886025:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826352.886028:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826352.888474:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226217:x1874663332662144:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 3607us (3844us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.888488:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68619 00000100:00100000:1.0:1787826352.892952:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826352.893037:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826352.893140:0:6541:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332662528 00000100:00100000:2.0:1787826352.893149:0:6541:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332662528 00000100:00100000:2.0:1787826352.893202:0:6541:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1235 00000100:00100000:2.0:1787826352.893211:0:6541:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226217:x1874663332662528:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:2.0:1787826352.893273:0:6541:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826352.893301:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4be19 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019c7 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.893319:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f7967ac8a00) 00010000:00010000:2.0:1787826352.893330:0:6541:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826352.893405:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e763800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305152:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826352.893456:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826352.893495:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826352.894934:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346305152 00000100:00100000:1.0:1787826352.894945:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346305152 00000100:00100000:1.0:1787826352.895005:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24007 00000100:00100000:1.0:1787826352.895017:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305152:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826352.895155:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305152:12345-0@lo:1000:mdt_io00_001.0 Request processed in 141us (1698us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826352.895163:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24007 00000100:00100000:3.0:1787826352.895180:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332662272 00000100:00100000:3.0:1787826352.895185:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332662272 00000100:00100000:3.0:1787826352.895240:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11181 00000100:00100000:3.0:1787826352.895248:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2393:x1874663332662272:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:3.0:1787826352.895259:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787826352.895260:0:6541:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957461 gen 2 00000004:00080000:0.0:1787826352.895266:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957458 committed transno 8589957461 00000004:00080000:0.0:1787826352.895286:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957459 committed transno 8589957461 00000004:00080000:0.0:1787826352.895300:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957460 committed transno 8589957461 00000004:00080000:0.0:1787826352.895311:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957461 committed transno 8589957461 00000100:00100000:0.0:1787826352.895328:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e763800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305152:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.895398:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305280:0@lo:1000:mdt_io00_001.0 00010000:00010000:3.0:1787826352.895398:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4bdfd lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019c0 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.895415:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f79785cdb00) 00010000:00010000:3.0:1787826352.895425:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826352.895433:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.895455:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826352.895463:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332662272:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 217us (2522us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.895468:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346305280 00000100:00100000:3.0:1787826352.895470:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11181 00000100:00100000:0.0:1787826352.895471:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346305280 00000100:00100000:0.0:1787826352.895494:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24008 00000100:00100000:0.0:1787826352.895501:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305280:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.895567:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305280:12345-0@lo:1000:mdt_io00_001.0 Request processed in 66us (134us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.895573:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24008 00000100:00100000:0.0:1787826352.895601:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305280:0@lo:1000:mdt_io00_001.0 00000004:00080000:0.0:1787826352.895719:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00000100:00100000:0.0:1787826352.895765:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305408:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.895790:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.895805:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826352.895968:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346305408 00000100:00100000:2.0:1787826352.895973:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346305408 00000100:00100000:2.0:1787826352.895997:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24009 00000100:00100000:2.0:1787826352.896004:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305408:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826352.896057:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305408:12345-0@lo:1000:mdt_io00_001.0 Request processed in 53us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.896064:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24009 00000100:00100000:0.0:1787826352.896621:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305408:0@lo:1000:mdt_io00_001.0 00000004:00080000:0.0:1787826352.896665:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00000100:00100000:0.0:1787826352.896689:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305536:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.896717:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.896735:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.896749:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346305536 00000100:00100000:0.0:1787826352.896754:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346305536 00000100:00100000:0.0:1787826352.896980:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24010 00000100:00100000:0.0:1787826352.896988:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305536:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.897048:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305536:12345-0@lo:1000:mdt_io00_001.0 Request processed in 60us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.897054:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24010 00000100:00100000:0.0:1787826352.897081:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305536:0@lo:1000:mdt_io00_001.0 00000004:00020000:0.0:1787826352.897181:0:6541:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:0.0:1787826352.897217:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.897243:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.897252:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00020000:1.0:1787826352.897303:0:11629:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4440 00010000:00010000:1.0:1787826352.897369:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897384:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897395:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.897423:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.897437:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 8 pid: 6527 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.897448:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346305664:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826352.897508:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318b40 going to sleep for 11 seconds 00000004:00200000:1.0:1787826352.897813:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e59:0x0]: returning size 6 00010000:00010000:1.0:1787826352.897904:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897912:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897918:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897924:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897930:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826352.897935:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897942:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897953:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7952ca9200) 00010000:00010000:1.0:1787826352.897956:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826352.897959:0:11629:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897972:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897978:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4bda2 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897991:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.897997:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.898003:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.898007:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826352.898010:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.898015:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.898021:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f79785cdd00) 00010000:00010000:1.0:1787826352.898023:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826352.898025:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.898030:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4bda9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.898077:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4bd9b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.898092:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.898099:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:1.0:1787826352.898101:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826352.898128:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826352.898156:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226215:x1874663332661504:12345-192.168.203.56@tcp:101:cat.0 Request processed in 560098us (560796us total) trans 4294985461 rc 0/0 00000100:00100000:1.0:1787826352.898164:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68614 00000100:00100000:0.0:1787826352.910662:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346305664:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826352.910699:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.912960:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826352.913009:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826352.913100:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332662656 00000100:00100000:1.0:1787826352.913143:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332662656 00010000:00010000:1.0:1787826352.913187:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 9 pid: 6527 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826352.913226:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22033 00000100:00100000:1.0:1787826352.913237:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7947dfdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2396:x1874663332662656:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826352.913251:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826352.913258:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 9 pid: 6527 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.913271:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 9 pid: 6527 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.913293:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4bd86 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801996 expref: 9 pid: 6527 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.913314:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7952ca9200) 00010000:00010000:1.0:1787826352.913319:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.913328:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.913336:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.913371:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826352.913426:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dfdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2396:x1874663332662656:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 190us (474us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826352.913434:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22033 00000100:00100000:2.0:1787826352.913715:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332662784 00000100:00100000:2.0:1787826352.913721:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332662784 00000100:00100000:2.0:1787826352.913774:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30872 00000100:00100000:2.0:1787826352.913782:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332662784:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826352.913796:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826352.913810:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4be35 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019ce expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826352.913859:0:35096:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:35096:1874663346305792:192.168.203.56@tcp:106: 00000100:00100000:2.0:1787826352.913916:0:35096:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362e40 going to sleep for 11 seconds 00000100:00080000:1.0:1787826352.913979:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787826352.914227:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.914236:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.914257:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.914266:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.914273:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.914306:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.914314:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.914323:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.914339:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305920:0@lo:101:mdt_io00_001.0 00000100:00100000:0.0:1787826352.914399:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.914422:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826352.914471:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346305920 00000100:00100000:1.0:1787826352.914476:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346305920 00000100:00100000:1.0:1787826352.914516:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24011 00000100:00100000:1.0:1787826352.914524:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:6541:x1874663346305920:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826352.914534:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826352.914548:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4be43 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.914566:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.914575:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 4/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4be43 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.914597:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.914609:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 8 pid: 35080 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826352.914619:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797cf2b800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346306048:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826352.914674:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826352.917435:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826352.917574:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797cf2b800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346306048:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826352.917614:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 3/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4be43 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.917628:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:2.0:1787826352.917639:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826352.917643:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 3/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4be43 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.917653:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:2.0:1787826352.917719:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6541:x1874663346305920:12345-0@lo:101:mdt_io00_001.0 Request processed in 3195us (3319us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.917726:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24011 00000100:00100000:1.0:1787826352.918043:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346305920:0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826352.918061:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.918071:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826352.918256:0:35096:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x2c0000400:0x4cd:0x0] updating lvb size: 0 -> 6 00002000:00010000:3.0:1787826352.918260:0:35096:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x2c0000400:0x4cd:0x0] updating lvb mtime: 0 -> 1787826346 00002000:00010000:3.0:1787826352.918263:0:35096:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x2c0000400:0x4cd:0x0] updating lvb atime: 0 -> 1787826346 00002000:00010000:3.0:1787826352.918265:0:35096:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x2c0000400:0x4cd:0x0] updating lvb ctime: 0 -> 1787826346 00002000:00010000:3.0:1787826352.918267:0:35096:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x2c0000400:0x4cd:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:3.0:1787826352.918285:0:35096:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:35096:1874663346305792:192.168.203.56@tcp:106: 00010000:00010000:3.0:1787826352.918314:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4be35 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ce expref: 6 pid: 35096 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.918334:0:35096:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a56ed3bb/0xe21309aa81f4be35 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ce expref: 6 pid: 35096 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.918346:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:3.0:1787826352.918411:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332662784:12345-192.168.203.56@tcp:101:cat.0 Request processed in 4630us (5401us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826352.918418:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30872 00000100:00100000:0.0:1787826352.919147:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332662912 00000100:00100000:0.0:1787826352.919156:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332662912 00010000:00010000:0.0:1787826352.919191:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 9 pid: 35080 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.919221:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22034 00000100:00100000:0.0:1787826352.919232:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332662912:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826352.919247:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826352.919254:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 9 pid: 35080 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.919266:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 9 pid: 35080 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.919283:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4bd94 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080199d expref: 9 pid: 35080 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.919297:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:0.0:1787826352.919303:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4be43 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.919312:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4be43 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.919321:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4be43 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.919353:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4be43 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.919388:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346306176:0@lo:105: 00000100:00100000:0.0:1787826352.919434:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.919470:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826352.921081:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346306176 00000100:00100000:2.0:1787826352.921092:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346306176 00000100:00100000:2.0:1787826352.921188:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7908 00000100:00100000:2.0:1787826352.921198:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346306176:12345-0@lo:105: 00010000:00010000:2.0:1787826352.921212:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.921222:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.921231:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.921238:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.921244:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.921252:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.921272:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42600) 00000100:00100000:2.0:1787826352.921341:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346306176:12345-0@lo:105: Request processed in 144us (1906us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.921349:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7908 00010000:00010000:1.0:1787826352.921601:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.921612:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826352.921648:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00010000:00010000:1.0:1787826352.921690:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.921702:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.921713:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826352.921731:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0f100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306304:0@lo:101:mdt_io00_001.0 00000100:00100000:1.0:1787826352.921769:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826352.921802:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb73c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826352.922014:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346306304 00000100:00100000:2.0:1787826352.922019:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346306304 00000100:00100000:2.0:1787826352.922056:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24012 00000100:00100000:2.0:1787826352.922063:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6541:x1874663346306304:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:2.0:1787826352.922079:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826352.922094:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4be58 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4be51 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.922142:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4be58 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4be51 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.922155:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4be58 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4be51 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.922162:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x4ec94fd6 (ffff9f7967ac8a00) 00010000:00010000:2.0:1787826352.922165:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826352.922168:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826352.922205:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306304:12345-0@lo:101:mdt_io00_001.0 Request processed in 142us (435us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.922210:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24012 00000100:00100000:3.0:1787826352.922289:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0f100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306304:0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826352.922305:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be58 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.922315:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be58 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826352.922331:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00000100:00100000:3.0:1787826352.922393:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306432:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.922423:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826352.922439:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.922545:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346306432 00000100:00100000:0.0:1787826352.922550:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346306432 00000100:00100000:0.0:1787826352.922585:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24013 00000100:00100000:0.0:1787826352.922592:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306432:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.922666:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306432:12345-0@lo:1000:mdt_io00_001.0 Request processed in 74us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.922673:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24013 00000100:00100000:0.0:1787826352.922701:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346306176:0@lo:105: 00010000:00010000:0.0:1787826352.922713:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826352.922756:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332662912:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3525us (5329us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.922765:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22034 00000100:00100000:0.0:1787826352.922902:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306432:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.922952:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306560:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.922978:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.922995:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.923007:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346306560 00000100:00100000:0.0:1787826352.923010:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346306560 00000100:00100000:0.0:1787826352.923031:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24014 00000100:00100000:0.0:1787826352.923038:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306560:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.923097:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306560:12345-0@lo:1000:mdt_io00_001.0 Request processed in 59us (117us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.923126:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24014 00000100:00100000:0.0:1787826352.923152:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306560:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.923189:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306688:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.923214:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.923229:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.923240:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346306688 00000100:00100000:0.0:1787826352.923244:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346306688 00000100:00100000:0.0:1787826352.923264:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24015 00000100:00100000:0.0:1787826352.923269:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306688:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.923388:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306688:12345-0@lo:1000:mdt_io00_001.0 Request processed in 117us (172us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826352.923395:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24015 00000100:00100000:0.0:1787826352.923420:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306688:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.923475:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306816:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.923498:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.923512:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.923522:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346306816 00000100:00100000:0.0:1787826352.923525:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346306816 00000100:00100000:0.0:1787826352.923545:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24016 00000100:00100000:0.0:1787826352.923562:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306816:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.923605:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306816:12345-0@lo:1000:mdt_io00_001.0 Request processed in 54us (107us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.923615:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24016 00000100:00100000:0.0:1787826352.923640:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306816:0@lo:1000:mdt_io00_001.0 00010000:00010000:0.0:1787826352.923745:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.923754:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.923764:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.923775:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306944:0@lo:101:mdt_io00_001.0 00000100:00100000:0.0:1787826352.923797:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.923811:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.923821:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346306944 00000100:00100000:0.0:1787826352.923824:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346306944 00000100:00100000:0.0:1787826352.923845:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24017 00000100:00100000:0.0:1787826352.923851:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6541:x1874663346306944:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826352.923859:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826352.923871:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4be66 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4be5f expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.923889:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4be66 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4be5f expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.923900:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4be66 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4be5f expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.923912:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:0.0:1787826352.923915:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1787826352.923918:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826352.923946:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346306944:12345-0@lo:101:mdt_io00_001.0 Request processed in 95us (148us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.923952:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24017 00000100:00100000:0.0:1787826352.923976:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346306944:0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826352.923988:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be66 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.923998:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be66 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826352.924010:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000403:0x1dc0:0x0] 00000100:00100000:0.0:1787826352.924038:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307072:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.924061:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.924073:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.924083:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346307072 00000100:00100000:0.0:1787826352.924087:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346307072 00000100:00100000:0.0:1787826352.924125:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24018 00000100:00100000:0.0:1787826352.924132:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346307072:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.924181:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346307072:12345-0@lo:1000:mdt_io00_001.0 Request processed in 50us (119us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.924188:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24018 00000100:00100000:0.0:1787826352.924212:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307072:0@lo:1000:mdt_io00_001.0 00000004:00080000:0.0:1787826352.924244:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00010000:00010000:0.0:1787826352.924273:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.924286:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.924293:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.924313:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.924325:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 7 pid: 11629 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.924336:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346307200:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826352.926477:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826352.930183:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826352.930382:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332663040 00000100:00100000:0.0:1787826352.930394:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332663040 00000100:00100000:0.0:1787826352.930552:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30873 00000100:00100000:0.0:1787826352.930574:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:226215:x1874663332663040:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826352.930593:0:17826:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826352.930629:0:17826:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019d5 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.930664:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 7 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.930680:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4cd:0x0].0x0 since=6s v=(0 0) 00010000:00010000:0.0:1787826352.930685:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826352.930691:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 4/0,0 mode: --/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.930726:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 7 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.930737:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 7 pid: 17826 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.930750:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346307328:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826352.930843:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826352.936668:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346307328:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826352.936714:0:17826:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 3/0,0 mode: --/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 6 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.936736:0:17826:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:2.0:1787826352.936923:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001_UUID:6541:1874663346307200:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826352.936941:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826352.938764:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226215:x1874663332663040:12345-192.168.203.56@tcp:101:cat.0 Request processed in 8191us (8580us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826352.938776:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30873 00000100:00100000:1.0:1787826352.945160:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826352.945208:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826352.945335:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332663168 00000100:00100000:0.0:1787826352.945345:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332663168 00010000:00010000:0.0:1787826352.945412:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 8 pid: 11629 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.945449:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332663296 00000100:00100000:0.0:1787826352.945452:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332663296 00010000:00010000:0.0:1787826352.945482:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 8 pid: 17826 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.945506:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22035 00000100:00100000:0.0:1787826352.945514:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332663168:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826352.945529:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826352.945536:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 8 pid: 11629 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945549:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 8 pid: 11629 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945572:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4be27 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e59:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019a4 expref: 8 pid: 11629 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945590:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:0.0:1787826352.945596:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945605:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945613:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945643:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826352.945687:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332663168:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 173us (536us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.945694:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22035 00000100:00100000:0.0:1787826352.945711:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22036 00000100:00100000:0.0:1787826352.945717:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332663296:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826352.945725:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826352.945745:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 8 pid: 17826 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945756:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 8 pid: 17826 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945772:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000430ce35c/0xe21309aa81f4bd0f lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e876080196c expref: 8 pid: 17826 timeout: 8265 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945784:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826352.945788:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826352.945795:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945804:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.945833:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.945847:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b306300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6511:1874663346307456:192.168.203.56@tcp:105: 00000100:00100000:0.0:1787826352.946013:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318240 going to sleep for 11 seconds 00000100:00080000:0.0:1787826352.946035:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787826352.946047:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.946055:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.952567:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.952585:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x2 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.952599:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x2 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.952640:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307584:0@lo:101:mdt_io00_001.0 00000100:00100000:0.0:1787826352.952716:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.952752:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826352.952821:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346307584 00000100:00100000:1.0:1787826352.952828:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346307584 00000100:00100000:1.0:1787826352.952904:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24019 00000100:00100000:1.0:1787826352.952915:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6541:x1874663346307584:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826352.952931:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826352.952951:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4be82 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4be7b expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.952978:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4be82 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4be7b expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.952992:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4be82 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4be7b expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826352.953000:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7967ac1200) 00010000:00010000:1.0:1787826352.953003:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826352.953007:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826352.953054:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346307584:12345-0@lo:101:mdt_io00_001.0 Request processed in 140us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826352.953060:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24019 00000100:00100000:3.0:1787826352.953750:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307584:0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826352.953769:0:6541:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 4/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.953785:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.953794:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826352.953928:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307712:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.953968:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826352.954033:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.959477:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826352.959550:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332663424 00000100:00100000:0.0:1787826352.959559:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332663424 00000100:00100000:0.0:1787826352.959623:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68620 00000100:00100000:1.0:1787826352.960762:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346307712 00000100:00100000:1.0:1787826352.960773:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346307712 00000100:00100000:1.0:1787826352.960827:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24020 00000100:00100000:1.0:1787826352.960898:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346307712:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826352.961029:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346307712:12345-0@lo:1000:mdt_io00_001.0 Request processed in 136us (7060us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826352.961037:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24020 00000100:00100000:0.0:1787826352.961223:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226215:x1874663332663424:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826352.961240:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826352.961262:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4be89 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e59:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019dc expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.961316:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4be89 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e59:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019dc expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.961330:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4be89 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e59:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019dc expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.961350:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:0.0:1787826352.961355:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787826352.961369:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826352.961416:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226215:x1874663332663424:12345-192.168.203.56@tcp:101:cat.0 Request processed in 202us (1952us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.961423:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68620 00000100:00100000:3.0:1787826352.962286:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b306300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6511:1874663346307456:192.168.203.56@tcp:105: 00010000:00010000:3.0:1787826352.962309:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826352.962355:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332663296:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 16637us (17145us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826352.962374:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22036 00000100:00100000:3.0:1787826352.962419:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307712:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.962493:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307840:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.962535:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826352.962565:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.963518:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346307840 00000100:00100000:0.0:1787826352.963528:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346307840 00000100:00100000:0.0:1787826352.963571:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24021 00000100:00100000:0.0:1787826352.963580:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346307840:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.963681:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346307840:12345-0@lo:1000:mdt_io00_001.0 Request processed in 102us (1145us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.963688:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24021 00000100:00100000:3.0:1787826352.963864:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307840:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826352.963896:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00000100:00100000:3.0:1787826352.963949:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307968:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.963977:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826352.964128:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346307968 00000100:00100000:1.0:1787826352.964134:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346307968 00000100:00100000:1.0:1787826352.964174:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24022 00000100:00100000:1.0:1787826352.964181:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346307968:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826352.966066:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346307968:12345-0@lo:1000:mdt_io00_001.0 Request processed in 1877us (2079us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826352.966078:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24022 00000100:00100000:3.0:1787826352.967077:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00ed80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826352.967132:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346307968:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.967213:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346308096:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.967261:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826352.967276:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826352.968315:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346308096 00000100:00100000:0.0:1787826352.968326:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346308096 00000100:00100000:0.0:1787826352.968398:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24023 00000100:00100000:0.0:1787826352.968409:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346308096:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.968519:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346308096:12345-0@lo:1000:mdt_io00_001.0 Request processed in 112us (1257us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.968526:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24023 00000100:00100000:3.0:1787826352.968999:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346308096:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.969101:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346308224:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826352.969267:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826352.969295:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826352.969350:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346308224 00000100:00100000:2.0:1787826352.969355:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346308224 00000100:00100000:2.0:1787826352.969408:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24024 00000100:00100000:2.0:1787826352.969417:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346308224:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826352.969489:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6541:x1874663346308224:12345-0@lo:1000:mdt_io00_001.0 Request processed in 74us (222us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826352.969495:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24024 00000100:00100000:3.0:1787826352.969561:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0001-mdtlov_UUID:6541:1874663346308224:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826352.970146:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826352.970212:0:20162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332663552 00000100:00100000:2.0:1787826352.970217:0:20162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332663552 00002000:00100000:2.0:1787826352.970252:0:20162:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x4cd:0x0] (0->4095) req@ffff9f784b305500 x1874663332663552/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:708/0 lens 488/0 e 0 to 0 dl 1787826363 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826352.970262:0:20162:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b305500 with x1874663332663552 ext(0->4095) 00002000:00010000:2.0:1787826352.970268:0:20162:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784b305500 00000100:00100000:2.0:1787826352.970296:0:20162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27663 00000100:00100000:2.0:1787826352.970304:0:20162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332663552:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:2.0:1787826352.970464:0:20162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332663552:12345-192.168.203.56@tcp:3:cat.0 Request processed in 160us (320us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1787826352.970471:0:20162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27663 00002000:00100000:2.0:1787826352.970477:0:20162:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x4cd:0x0] (0->4095) req@ffff9f784b305500 x1874663332663552/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:708/0 lens 488/4536 e 0 to 0 dl 1787826363 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826352.970485:0:20162:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b305500 with x1874663332663552 ext(0->4095) 00002000:00010000:2.0:1787826352.970488:0:20162:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784b305500 00000004:00080000:2.0:1787826352.972382:0:6541:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x240000403:0x1e59:0x0] is inserted into orphan list, open count = 1 00000040:00080000:2.0:1787826352.972456:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1224, 1240 off1117520 00000040:00080000:2.0:1787826352.972518:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1542, 1320 off1560936 00000004:00080000:2.0:1787826352.972525:0:6541:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d00e9c0 with version 6876 00000100:00100000:1.0:1787826352.976772:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826352.977155:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332663808 00000100:00100000:3.0:1787826352.977170:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332663808 00000004:00080000:0.0:1787826352.977178:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6876 rpc_version 6876 00000020:00100000:0.0:1787826352.977219:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826352.977232:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b304000 x1874663346308480/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826352.977258:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346308480:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826352.977291:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9382 00000100:00100000:3.0:1787826352.977305:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226215:x1874663332663808:12345-192.168.203.56@tcp:35:cat.0 00000100:00100000:0.0:1787826352.977321:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826352.977535:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318840 going to sleep for 11 seconds 00000004:00080000:3.0:1787826352.977547:0:6531:0:(mdd_object.c:4014:mdd_close()) Object [0x240000403:0x1e59:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000100:00100000:1.0:1787826352.977558:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346308480 00000100:00100000:1.0:1787826352.977562:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346308480 00000100:00100000:1.0:1787826352.977601:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24025 00000100:00100000:1.0:1787826352.977608:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346308480:12345-0@lo:1000:osp_up0-1.0 00000040:00080000:3.0:1787826352.977791:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].225, 64 off22592 00000004:00080000:3.0:1787826352.977805:0:6531:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8165189698296ns, next at 8166189695583ns, rc = 0 00010000:00010000:3.0:1787826352.977969:0:6531:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6531 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826352.977971:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346308480:12345-0@lo:1000:osp_up0-1.0 Request processed in 363us (650us total) trans 8589957462 rc 0/0 00000100:00100000:1.0:1787826352.977979:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24025 00010000:00010000:3.0:1787826352.977985:0:6531:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010080000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.977996:0:6531:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010080000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.978005:0:6531:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010080000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.978015:0:6531:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010480000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.978022:0:6531:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010480000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826352.978044:0:6531:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826352.978128:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346308480:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826352.978133:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226215:x1874663332663808:12345-192.168.203.56@tcp:35:cat.0 Request processed in 829us (1368us total) trans 4294985463 rc 0/0 00000100:00100000:3.0:1787826352.978142:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9382 00010000:00010000:2.0:1787826352.978180:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978189:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978196:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978202:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826352.978207:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978216:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978227:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:2.0:1787826352.978231:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826352.978234:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978240:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4be6d lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e59:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826352.978256:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000403:0x1dc0:0x0] 00010000:00010000:2.0:1787826352.978263:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be66 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978271:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be66 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978290:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978296:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978301:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978305:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826352.978308:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978314:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978321:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7952ca9200) 00010000:00010000:2.0:1787826352.978324:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826352.978327:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978334:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43a00/0xe21309aa81f4be20 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826352.978350:0:6541:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d6a87d70 reply state 00000000ea50d23a transno 4294985462 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978370:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978377:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f784b4a5a00) 00010000:00010000:2.0:1787826352.978380:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826352.978384:0:6541:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:2.0:1787826352.978387:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00010000:00010000:2.0:1787826352.978392:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978399:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978406:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978425:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826352.978430:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00010000:00010000:2.0:1787826352.978435:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be58 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978442:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4be58 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978451:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978456:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978463:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826352.978478:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:2.0:1787826352.978519:0:6541:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226217:x1874663332662528:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 85308us (85480us total) trans 4294985462 rc 0/0 00000100:00100000:2.0:1787826352.978526:0:6541:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1235 00010000:00010000:0.0:1787826352.978624:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787826352.978630:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978637:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826352.978640:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978646:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978656:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:0.0:1787826352.978658:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826352.978661:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978667:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4be90 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e59:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6531 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978678:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826352.978681:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978687:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826352.978691:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826352.978709:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978736:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978743:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826352.978771:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978779:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4be43 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be4a expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978795:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826352.978799:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978805:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826352.978808:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826352.978816:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978844:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978851:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826352.978861:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826352.978870:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 1/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978879:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43200/0xe21309aa81f4be7b lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be82 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.978897:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826352.978899:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826352.978901:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826352.978909:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826352.978921:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346308608:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826352.978960:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.978997:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346308736:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826352.979014:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346308608 00000100:00100000:1.0:1787826352.979021:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346308608 00000100:00100000:0.0:1787826352.979025:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826352.979069:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346308736 00000100:00100000:0.0:1787826352.979074:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346308736 00000100:00100000:1.0:1787826352.979075:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11182 00000100:00100000:1.0:1787826352.979087:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346308608:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826352.979099:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826352.979142:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4be4a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4be43 expref: 13 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826352.979149:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11183 00000100:00100000:0.0:1787826352.979162:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346308736:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826352.979163:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:0.0:1787826352.979175:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826352.979176:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826352.979191:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4be82 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4be7b expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826352.979203:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7967ac1200) 00010000:00010000:0.0:1787826352.979212:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826352.979221:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346308608:12345-0@lo:103:ldlm_bl.0 Request processed in 134us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826352.979228:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11182 00000100:00100000:0.0:1787826352.979244:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346308736:12345-0@lo:103:ldlm_bl.0 Request processed in 85us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826352.979251:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11183 00000100:00100000:0.0:1787826352.979279:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346308736:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826352.979304:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346308608:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826352.980971:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826353.324235:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826354.065732:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.065841:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332663936 00000100:00100000:1.0:1787826354.065851:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332663936 00000100:00100000:1.0:1787826354.065934:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68621 00000100:00100000:1.0:1787826354.065947:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332663936:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826354.065972:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.065996:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4be97 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019e3 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.066068:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4be9e lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.066081:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4be9e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.066094:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4be9e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826354.066227:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e58:0x0]: returning size 4096 00000004:00200000:1.0:1787826354.066232:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787826354.066251:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4be9e lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.066277:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4be97 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019e3 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.066296:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4be9e lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019e3 expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.066304:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:1.0:1787826354.066307:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826354.066310:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826354.066377:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226253:x1874663332663936:12345-192.168.203.56@tcp:101:rm.0 Request processed in 434us (654us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.066387:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68621 00000100:00080000:0.0:1787826354.066544:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826354.072704:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826354.072820:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332664064 00000100:00100000:2.0:1787826354.072830:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332664064 00000100:00100000:2.0:1787826354.072904:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68622 00000100:00100000:2.0:1787826354.072917:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226253:x1874663332664064:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826354.072938:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826354.072963:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4bea5 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019ea expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.073025:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.073040:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.073052:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826354.073099:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19a:0x0]: returning size 4096 00000004:00200000:2.0:1787826354.073132:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826354.073182:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4bea5 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.073201:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.073216:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7967ac8d00) 00010000:00010000:2.0:1787826354.073219:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826354.073224:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826354.073280:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226253:x1874663332664064:12345-192.168.203.56@tcp:101:rm.0 Request processed in 367us (586us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826354.073286:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68622 00000100:00100000:1.0:1787826354.081888:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.082086:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332664192 00000100:00100000:3.0:1787826354.082097:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332664192 00000100:00080000:1.0:1787826354.082311:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826354.082333:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68623 00000100:00100000:3.0:1787826354.082345:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226253:x1874663332664192:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826354.082374:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.082398:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4beb3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019f1 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.082465:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 3/1,0 mode: --/PR res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.082478:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.082492:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826354.082717:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x198:0x0]: returning size 4096 00000004:00200000:3.0:1787826354.082723:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826354.082763:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4beb3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x198:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.082779:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.082788:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:3.0:1787826354.082792:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826354.082795:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826354.082862:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851834000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332664192:12345-192.168.203.56@tcp:101:rm.0 Request processed in 520us (983us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.082870:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68623 00000100:00100000:2.0:1787826354.086440:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.086518:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332664448 00000100:00100000:3.0:1787826354.086527:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332664448 00000100:00100000:3.0:1787826354.086600:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68624 00000100:00100000:3.0:1787826354.086611:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226253:x1874663332664448:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826354.086631:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.086654:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bec1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019f8 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826354.086806:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e58:0x0]: returning size 4096 00010000:00010000:3.0:1787826354.087201:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bec1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019f8 expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.087217:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826354.087220:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826354.087225:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bec1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019f8 expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.087243:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826354.087302:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226253:x1874663332664448:12345-192.168.203.56@tcp:101:rm.0 Request processed in 693us (872us total) trans 4294985464 rc 301/301 00000100:00100000:3.0:1787826354.087309:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68624 00010000:00010000:0.0:1787826354.091429:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4be58 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826354.091449:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4be58 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.091495:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be58 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.091501:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826354.091566:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:0.0:1787826354.091711:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf000/0xe21309aa81f4be51 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be58 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.091736:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4be66 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826354.091743:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4be66 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.091763:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be66 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.091768:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826354.091794:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4be5f lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4be66 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826354.091808:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985464 is committed 00000004:00080000:0.0:1787826354.091817:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 407 committed 00000001:00080000:0.0:1787826354.091840:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985462 is committed 00000001:00080000:0.0:1787826354.091854:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985460 is committed 00000020:00080000:0.0:1787826354.091994:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3437 batchid updated 00010000:00010000:0.0:1787826354.092079:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.092088:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.092095:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.092100:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826354.092129:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.092136:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.092149:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f784b4a5a00) 00010000:00010000:0.0:1787826354.092152:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826354.092157:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.092162:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4be3c lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787826354.092189:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3437 00000004:00080000:0.0:1787826354.092245:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846318d80 with version 6877 00000020:00080000:0.0:1787826354.092256:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3434 cancel update log [0x200000401:0x3:0x0].1538: rc = 0 00000020:00080000:0.0:1787826354.092325:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3434 cancel update log [0x240000400:0x3:0x0].1219: rc = 0 00000004:00080000:0.0:1787826354.092347:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846318a80 with version 6878 00000020:00080000:0.0:1787826354.092361:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3435 cancel update log [0x200000401:0x3:0x0].1539: rc = 0 00000020:00080000:0.0:1787826354.092382:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3435 cancel update log [0x240000400:0x3:0x0].1220: rc = 0 00000004:00080000:0.0:1787826354.092405:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846318780 with version 6879 00000020:00080000:0.0:1787826354.092407:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3436 cancel update log [0x200000401:0x3:0x0].1540: rc = 0 00000020:00080000:0.0:1787826354.092425:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3436 cancel update log [0x240000400:0x3:0x0].1221: rc = 0 00000004:00080000:0.0:1787826354.092444:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846318540 with version 6880 00000020:00080000:0.0:1787826354.092447:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3437 cancel update log [0x200000401:0x3:0x0].1541: rc = 0 00000020:00080000:0.0:1787826354.092463:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3437 cancel update log [0x240000400:0x3:0x0].1223: rc = 0 00000020:00080000:0.0:1787826354.092469:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3437 00000100:00100000:1.0:1787826354.093668:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346308992:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826354.093745:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.093789:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826354.093797:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346309120:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1787826354.093825:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.093861:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346308992 00000100:00100000:1.0:1787826354.093867:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346308992 00000100:00100000:1.0:1787826354.094090:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11184 00000100:00100000:1.0:1787826354.094100:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346308992:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787826354.094131:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826354.094157:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4be66 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4be5f expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.094175:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:1.0:1787826354.094186:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826354.094221:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346308992:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 123us (476us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.094227:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11184 00000100:00100000:1.0:1787826354.094253:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346309120 00000100:00100000:1.0:1787826354.094257:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346309120 00000100:00100000:1.0:1787826354.094291:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30874 00000100:00100000:1.0:1787826354.094298:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346309120:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:1.0:1787826354.094320:0:35352:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1229 count 1 00010000:00010000:1.0:1787826354.094346:0:35352:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.094380:0:35352:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.094389:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=1s v=(1 0) 00010000:00010000:1.0:1787826354.094393:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826354.094397:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.094427:0:35352:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 5 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.094441:0:35352:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 5 pid: 17826 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.094453:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346309248:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826354.094518:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826354.095662:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346308864:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787826354.095714:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1787826354.095764:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6877 rpc_version 6877 00000020:00100000:2.0:1787826354.095788:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826354.095797:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e761f80 x1874663346309504/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826354.095815:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346309504:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826354.095847:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.095866:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a780 going to sleep for 11 seconds 00000100:00080000:1.0:1787826354.096130:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826354.096257:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.096430:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346308864 00000100:00100000:0.0:1787826354.096438:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346308864 00000100:00100000:0.0:1787826354.096488:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11185 00000100:00100000:0.0:1787826354.096497:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7947dffc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3652:x1874663346308864:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787826354.096508:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.096532:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4be58 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x198:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4be51 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.096550:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x4ec94fd6 (ffff9f7967ac8a00) 00010000:00010000:0.0:1787826354.096560:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826354.096601:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dffc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346308864:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 106us (886us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.096608:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11185 00000100:00100000:2.0:1787826354.096662:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346309504 00000100:00100000:2.0:1787826354.096666:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346309504 00000100:00100000:2.0:1787826354.096703:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24026 00000100:00100000:2.0:1787826354.096710:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346309504:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826354.096869:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346309504:12345-0@lo:1000:osp_up0-1.0 Request processed in 159us (1020us total) trans 8589957463 rc 0/0 00000100:00100000:0.0:1787826354.096927:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346309504:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1787826354.096939:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6878 rpc_version 6878 00000020:00100000:0.0:1787826354.096953:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826354.096958:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c91180 x1874663346309760/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826354.096974:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346309760:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826354.097004:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.097020:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.097284:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346308992:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826354.097477:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332664576 00000100:00100000:1.0:1787826354.097482:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332664576 00000100:00100000:1.0:1787826354.097530:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9383 00000100:00100000:2.0:1787826354.098003:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24026 00000100:00100000:2.0:1787826354.098022:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346309760 00000100:00100000:2.0:1787826354.098026:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346309760 00000100:00100000:2.0:1787826354.098053:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24027 00000100:00100000:2.0:1787826354.098061:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346309760:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826354.098195:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346309760:12345-0@lo:1000:osp_up0-1.0 Request processed in 135us (1190us total) trans 8589957464 rc 0/0 00000100:00100000:2.0:1787826354.098202:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24027 00000100:00100000:2.0:1787826354.098254:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346309760:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826354.098264:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6879 rpc_version 6879 00000020:00100000:2.0:1787826354.098279:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826354.098285:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e762680 x1874663346310016/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826354.098302:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346310016:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826354.098334:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.098368:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826354.098394:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346308864:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826354.098632:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332664576:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:0.0:1787826354.099371:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346310016 00000100:00100000:0.0:1787826354.099380:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346310016 00000100:00100000:0.0:1787826354.099419:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24028 00000100:00100000:0.0:1787826354.099427:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346310016:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826354.099573:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346310016:12345-0@lo:1000:osp_up0-1.0 Request processed in 146us (1237us total) trans 8589957465 rc 0/0 00000100:00100000:0.0:1787826354.099580:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24028 00000100:00100000:2.0:1787826354.100831:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346310016:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826354.100848:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6880 rpc_version 6880 00000020:00100000:2.0:1787826354.102075:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826354.102088:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784a91c380 x1874663346310272/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826354.102145:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346310272:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826354.102210:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.102243:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826354.102271:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346310272 00000100:00100000:2.0:1787826354.102277:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346310272 00000100:00100000:2.0:1787826354.102325:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24029 00000100:00100000:2.0:1787826354.102334:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346310272:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826354.103678:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346310272:12345-0@lo:1000:osp_up0-1.0 Request processed in 1338us (1461us total) trans 8589957466 rc 0/0 00000100:00100000:2.0:1787826354.103691:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24029 00000100:00100000:1.0:1787826354.103805:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91c380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346310272:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826354.105092:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346309248:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826354.105141:0:35352:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.105416:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.105688:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332664704 00000100:00100000:0.0:1787826354.105696:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332664704 00010000:00010000:0.0:1787826354.105787:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 6 pid: 17826 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.105829:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22037 00000100:00100000:0.0:1787826354.105838:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332664704:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826354.105853:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.105858:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 6 pid: 17826 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.106311:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 6 pid: 17826 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.106361:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4be74 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e87608019d5 expref: 6 pid: 17826 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.106378:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826354.106382:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4cd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826354.106388:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.106398:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.106426:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826354.106478:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332664704:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 641us (1062us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.106487:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22037 00010000:00010000:0.0:1787826354.110294:0:35352:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.110313:0:35352:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.110325:0:35352:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.110333:0:35352:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.110364:0:35352:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826354.110558:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346309120:12345-0@lo:6:osp-syn-1-1.0 Request processed in 16258us (16729us total) trans 4294976169 rc 0/0 00000100:00100000:0.0:1787826354.110568:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30874 00000004:00080000:0.0:1787826354.110631:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7842c91f80/2, rc 0, transno 4294976169 00000100:00100000:0.0:1787826354.110637:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346309120:0@lo:6:osp-syn-1-1.0 00000100:00080000:1.0:1787826354.110670:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787826354.110683:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826354.110691:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.110699:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826354.110703:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.110712:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.110730:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.110737:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e826c654/0xe21309aa81f4becf lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4cd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.110759:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826354.110761:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826354.110762:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826354.112402:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332664576:12345-192.168.203.56@tcp:37:rm.0 Request processed in 13774us (16135us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.112415:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9383 00000100:00100000:0.0:1787826354.124223:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.124343:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332664832 00000100:00100000:3.0:1787826354.124364:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332664832 00000100:00100000:3.0:1787826354.124422:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9384 00000100:00100000:3.0:1787826354.124434:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943030c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332664832:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:0.0:1787826354.131122:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943030c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332664832:12345-192.168.203.56@tcp:37:rm.0 Request processed in 6666us (6883us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.131137:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9384 00000100:00100000:1.0:1787826354.151134:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.159063:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332665088 00000100:00100000:1.0:1787826354.159078:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332665088 00000100:00100000:1.0:1787826354.159162:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9385 00000100:00100000:1.0:1787826354.159174:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332665088:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:1.0:1787826354.159451:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1225, 344 off1117864 00000040:00080000:1.0:1787826354.159505:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1543, 424 off1561360 00000004:00080000:1.0:1787826354.159511:0:17601:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784afb7a80 with version 6881 00000004:00080000:1.0:1787826354.159526:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6881 rpc_version 6881 00000020:00100000:1.0:1787826354.159554:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826354.159563:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7847706680 x1874663346310528/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826354.159588:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346310528:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826354.159663:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.159698:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.159979:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346310528 00000100:00100000:0.0:1787826354.159985:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346310528 00000100:00100000:0.0:1787826354.160034:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24030 00000100:00100000:0.0:1787826354.160042:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346310528:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826354.160217:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346310528:12345-0@lo:1000:osp_up0-1.0 Request processed in 176us (553us total) trans 8589957467 rc 0/0 00000100:00100000:0.0:1787826354.160225:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24030 00000100:00100000:0.0:1787826354.160281:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346310528:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826354.160387:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226253:x1874663332665088:12345-192.168.203.56@tcp:35:rm.0 Request processed in 1215us (9262us total) trans 4294985465 rc 0/0 00000100:00100000:0.0:1787826354.160395:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9385 00000100:00100000:3.0:1787826354.165303:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.165344:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332665344 00000100:00100000:3.0:1787826354.165364:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332665344 00000100:00100000:3.0:1787826354.165420:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68625 00000100:00100000:3.0:1787826354.165431:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226253:x1874663332665344:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826354.165455:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.165480:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bed6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e87608019ff expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.165640:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bedd lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.165653:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bedd lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.165664:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bedd lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826354.165681:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e58:0x0]: returning size 4096 00010000:00010000:3.0:1787826354.165778:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4bed6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ff expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.165800:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bedd lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ff expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.165809:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:3.0:1787826354.165812:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1787826354.165815:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826354.166979:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226253:x1874663332665344:12345-192.168.203.56@tcp:101:rm.0 Request processed in 443us (578us total) trans 4294985466 rc 0/0 00000100:00100000:3.0:1787826354.166998:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68625 00000100:00100000:1.0:1787826354.177950:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.178867:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332665472 00000100:00100000:0.0:1787826354.178878:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332665472 00000100:00100000:0.0:1787826354.178949:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68626 00000100:00100000:0.0:1787826354.178960:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332665472:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:0.0:1787826354.179011:0:6527:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:0.0:1787826354.179080:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332665472:12345-192.168.203.56@tcp:41:rm.0 Request processed in 122us (1140us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.179088:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68626 00000100:00100000:3.0:1787826354.184239:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.184283:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332665600 00000100:00100000:3.0:1787826354.184294:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332665600 00000100:00100000:3.0:1787826354.184359:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68627 00000100:00100000:3.0:1787826354.184370:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226253:x1874663332665600:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826354.184396:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.184419:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4beeb lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a06 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184493:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184506:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184518:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184533:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184540:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184546:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184660:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bf00 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184669:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bf00 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184676:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bf00 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826354.184695:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bf00 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184708:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184712:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184717:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184724:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184728:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826354.184731:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184736:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184747:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7967ac8d00) 00010000:00010000:3.0:1787826354.184750:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826354.184754:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184760:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4bef2 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184773:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184780:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184785:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184788:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826354.184790:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184796:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184801:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f79785cc000) 00010000:00010000:3.0:1787826354.184802:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826354.184804:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184807:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4bef9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184830:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4beeb lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a06 expref: 14 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184838:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bf00 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a06 expref: 13 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.184843:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826354.184845:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826354.184846:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826354.184910:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226253:x1874663332665600:12345-192.168.203.56@tcp:101:rm.0 Request processed in 542us (681us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.184918:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68627 00000100:00100000:2.0:1787826354.188644:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.188720:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332665728 00000100:00100000:1.0:1787826354.188730:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332665728 00000100:00100000:1.0:1787826354.188793:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68628 00000100:00100000:1.0:1787826354.188804:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226253:x1874663332665728:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826354.188823:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.188847:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4bf07 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a0d expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.188912:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bf0e lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.188926:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bf0e lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.188941:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bf0e lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826354.188995:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dc0:0x0]: returning size 6 00000004:00200000:1.0:1787826354.189000:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:1.0:1787826354.189039:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4bf07 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a0d expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.189055:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bf0e lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a0d expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.189064:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826354.189068:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826354.189071:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826354.189415:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226253:x1874663332665728:12345-192.168.203.56@tcp:101:rm.0 Request processed in 612us (778us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.189422:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68628 00000100:00100000:1.0:1787826354.201573:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.201624:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.201711:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332666112 00000100:00100000:0.0:1787826354.201722:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332666112 00000100:00100000:0.0:1787826354.201792:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68629 00000100:00100000:0.0:1787826354.201804:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226253:x1874663332666112:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826354.201859:0:6529:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.201889:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4bf0e lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a0d expref: 10 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.201909:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:0.0:1787826354.201920:0:6529:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826354.202003:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346310656:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826354.202061:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.202092:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.202156:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346310656 00000100:00100000:0.0:1787826354.202160:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346310656 00000100:00100000:0.0:1787826354.202193:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24031 00000100:00100000:0.0:1787826354.202200:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6529:x1874663346310656:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826354.202274:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6529:x1874663346310656:12345-0@lo:1000:mdt00_002.0 Request processed in 75us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.202282:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24031 00000100:00100000:0.0:1787826354.202323:0:6529:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985460 gen 2 00000004:00080000:0.0:1787826354.202329:0:6529:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985460 committed transno 4294985460 00000100:00100000:0.0:1787826354.202367:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346310656:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1787826354.202425:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.202435:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.202444:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.202456:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346310784:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826354.202481:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.202496:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.202508:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346310784 00000100:00100000:0.0:1787826354.202512:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346310784 00000100:00100000:0.0:1787826354.202536:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24032 00000100:00100000:0.0:1787826354.202543:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6529:x1874663346310784:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787826354.202552:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826354.202569:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4bf1c lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bf15 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.202593:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.202602:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4bf1c lrc: 4/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bf15 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.202626:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.202639:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 12 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.202649:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346310912:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826354.202710:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826354.202991:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332665856 00000100:00100000:3.0:1787826354.202996:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332665856 00000100:00100000:3.0:1787826354.203097:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11186 00000100:00100000:3.0:1787826354.203126:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:2393:x1874663332665856:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826354.203152:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826354.203169:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68400/0xe21309aa81f4bf00 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a06 expref: 13 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.203182:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f79785cc400) 00010000:00010000:3.0:1787826354.203190:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826354.203222:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332665856:12345-192.168.203.56@tcp:103:rm.0 Request processed in 118us (1597us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.203228:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11186 00000100:00100000:1.0:1787826354.206717:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826354.206914:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332666240 00000100:00100000:2.0:1787826354.206924:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332666240 00010000:00010000:2.0:1787826354.206969:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 12 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.207012:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22038 00000100:00100000:2.0:1787826354.207022:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2396:x1874663332666240:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826354.207039:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826354.207046:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 12 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.207059:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 12 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.207082:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7967ac8d00) 00010000:00010000:2.0:1787826354.207090:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4bf1c lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bf15 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.207100:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4bf1c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4bf15 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.207136:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4bf1c lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4bf15 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.207222:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4bf1c lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf15 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.207243:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001_UUID:125406:1874663346311040:0@lo:105: 00000100:00100000:2.0:1787826354.207298:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.207345:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00010000:00010000:0.0:1787826354.207696:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4beac lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ea expref: 12 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.207714:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:8152:1874663346310912:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826354.207808:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4bf1c lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf15 expref: 6 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.207819:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7967ac8d00) 00010000:00010000:0.0:1787826354.207821:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826354.207824:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826354.208060:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6529:x1874663346310784:12345-0@lo:101:mdt00_002.0 Request processed in 5515us (5576us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.208068:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24032 00000100:00100000:1.0:1787826354.208288:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346310784:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826354.208302:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.208310:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.208709:0:6510:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826354.208728:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346311040 00000100:00100000:3.0:1787826354.208733:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346311040 00000100:00100000:3.0:1787826354.208784:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7909 00000100:00100000:3.0:1787826354.208795:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b035c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:125406:x1874663346311040:12345-0@lo:105: 00010000:00010000:3.0:1787826354.208806:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.208815:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.208823:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.208831:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.208840:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.208849:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.208877:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42800) 00000100:00100000:3.0:1787826354.208930:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b035c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:125406:x1874663346311040:12345-0@lo:105: Request processed in 138us (1631us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.208937:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7909 00000100:00100000:3.0:1787826354.209001:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001_UUID:125406:1874663346311040:0@lo:105: 00010000:00010000:3.0:1787826354.209018:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826354.209066:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2396:x1874663332666240:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2043us (2356us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.209075:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22038 00010000:00010000:0.0:1787826354.209335:0:6529:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.209347:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826354.209390:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19a:0x0] 00010000:00010000:0.0:1787826354.209432:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.209440:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.209450:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.209469:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311168:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787826354.209507:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.209540:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826354.209734:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346311168 00000100:00100000:2.0:1787826354.209740:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346311168 00000100:00100000:2.0:1787826354.209777:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24033 00000100:00100000:2.0:1787826354.209786:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6529:x1874663346311168:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787826354.209800:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826354.209817:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4bf2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bf23 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.209838:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4bf2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bf23 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.209850:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4bf2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf23 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.209858:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f7967ac8600) 00010000:00010000:2.0:1787826354.209861:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826354.209864:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eea00, rc 0) 00000100:00100000:2.0:1787826354.209910:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6529:x1874663346311168:12345-0@lo:101:mdt00_002.0 Request processed in 125us (401us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826354.209916:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24033 00000100:00100000:1.0:1787826354.210013:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311168:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826354.210027:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf2a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.210036:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf2a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826354.210050:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19a:0x0] 00000100:00100000:1.0:1787826354.210088:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311296:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826354.210318:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.210345:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.210711:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346311296 00000100:00100000:0.0:1787826354.210717:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346311296 00000100:00100000:0.0:1787826354.210751:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24034 00000100:00100000:0.0:1787826354.210758:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6529:x1874663346311296:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826354.210836:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6529:x1874663346311296:12345-0@lo:1000:mdt00_002.0 Request processed in 78us (517us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.210843:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24034 00000100:00100000:2.0:1787826354.211069:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311296:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826354.211298:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311424:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787826354.211336:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.211371:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.211544:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346311424 00000100:00100000:0.0:1787826354.211550:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346311424 00000100:00100000:0.0:1787826354.211585:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24035 00000100:00100000:0.0:1787826354.211593:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6529:x1874663346311424:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826354.211870:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6529:x1874663346311424:12345-0@lo:1000:mdt00_002.0 Request processed in 276us (531us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.211878:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24035 00000100:00100000:1.0:1787826354.211927:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311424:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826354.211978:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5b800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311552:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787826354.212007:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.212060:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.212098:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346311552 00000100:00100000:0.0:1787826354.212391:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346311552 00000100:00100000:0.0:1787826354.212465:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24036 00000100:00100000:0.0:1787826354.212473:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6529:x1874663346311552:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787826354.212578:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6529:x1874663346311552:12345-0@lo:1000:mdt00_002.0 Request processed in 105us (569us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826354.212584:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24036 00000100:00100000:3.0:1787826354.212800:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5b800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311552:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1787826354.212871:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.212883:0:6529:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.212893:0:6529:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.212907:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977e0f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311680:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787826354.212939:0:6529:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826354.213063:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.213190:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346311680 00000100:00100000:1.0:1787826354.213196:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346311680 00000100:00100000:1.0:1787826354.213263:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24037 00000100:00100000:1.0:1787826354.213271:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6529:x1874663346311680:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826354.213282:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.213300:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4bf38 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bf31 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.213324:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4bf38 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bf31 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.213342:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4bf38 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf31 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.213360:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826354.213363:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826354.213367:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:1.0:1787826354.213404:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6529:x1874663346311680:12345-0@lo:101:mdt00_002.0 Request processed in 135us (465us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.213410:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24037 00000100:00100000:1.0:1787826354.213558:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977e0f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6529:1874663346311680:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787826354.213572:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.213584:0:6529:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.213611:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.213630:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.213639:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787826354.213958:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].216, 64 off22016 00000004:00080000:1.0:1787826354.213971:0:6529:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8166425865103ns, next at 8167425862631ns, rc = 0 00000040:00080000:1.0:1787826354.214005:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].942, 384 off886360 00000040:00080000:1.0:1787826354.214054:0:6529:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].945, 464 off986144 00000004:00080000:1.0:1787826354.214060:0:6529:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f78434e4240 with version 1837 00000004:00080000:3.0:1787826354.214297:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1837 rpc_version 1837 00000020:00100000:3.0:1787826354.214323:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826354.214332:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784836f800 x1874663346311936/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1664/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826354.214364:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346311936:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826354.214413:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826354.214433:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.214614:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346311936 00000100:00100000:1.0:1787826354.214630:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346311936 00000100:00100000:1.0:1787826354.214672:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24038 00000100:00100000:1.0:1787826354.214681:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346311936:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826354.214943:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346311936:12345-0@lo:1000:osp_up1-0.0 Request processed in 263us (529us total) trans 4294985467 rc 0/0 00000100:00100000:1.0:1787826354.214950:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24038 00000100:00100000:1.0:1787826354.215003:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346311936:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787826354.215037:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215048:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215058:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215066:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215072:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215078:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215097:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826354.215150:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215158:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215164:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215170:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826354.215175:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215191:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215203:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:1.0:1787826354.215206:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826354.215213:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215219:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf3f lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215232:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215240:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215246:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215263:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787826354.215275:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19a:0x0] 00010000:00010000:1.0:1787826354.215282:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215288:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215294:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215306:0:6529:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787826354.215310:0:6529:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19a:0x0] 00010000:00010000:1.0:1787826354.215314:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf2a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215320:0:6529:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf2a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.215394:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226253:x1874663332666112:12345-192.168.203.56@tcp:36:rm.0 Request processed in 13591us (13830us total) trans 8589957468 rc 0/0 00000100:00100000:1.0:1787826354.215401:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68629 00010000:00010000:1.0:1787826354.215423:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787826354.215429:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215435:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecea00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826354.215439:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215445:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215454:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:1.0:1787826354.215456:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1787826354.215459:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215465:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf46 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dc0:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215476:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826354.215480:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215485:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826354.215489:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826354.215496:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215519:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215525:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826354.215545:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 1/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215557:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68400/0xe21309aa81f4bf31 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf38 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215569:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826354.215572:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215579:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826354.215582:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826354.215589:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215603:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215609:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826354.215615:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215622:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42800/0xe21309aa81f4bf15 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf1c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215634:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826354.215635:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826354.215637:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826354.215648:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346312192:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826354.215680:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.215705:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826354.215709:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346312064:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826354.215726:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.215749:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346312192 00000100:00100000:1.0:1787826354.215754:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346312192 00000100:00100000:1.0:1787826354.215784:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346312064 00000100:00100000:1.0:1787826354.215787:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346312064 00000100:00100000:1.0:1787826354.215808:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11187 00000100:00100000:1.0:1787826354.215815:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3653:x1874663346312192:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826354.215825:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826354.215840:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42400/0xe21309aa81f4bf1c lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bf15 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215850:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7967ac8d00) 00010000:00010000:1.0:1787826354.215857:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826354.215881:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346312192:12345-0@lo:103:ldlm_bl.0 Request processed in 67us (200us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.215887:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11187 00000100:00100000:1.0:1787826354.215899:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11188 00000100:00100000:1.0:1787826354.215904:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346312064:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826354.215911:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826354.215922:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4bf38 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dc0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bf31 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.215931:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc0:0x0].0x0 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826354.215937:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826354.215958:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346312064:12345-0@lo:103:ldlm_bl.0 Request processed in 53us (230us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.215964:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11188 00000100:00100000:1.0:1787826354.215991:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346312064:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826354.216000:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346312192:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826354.216177:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826354.216485:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826354.228630:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.228683:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.228996:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332666368 00000100:00100000:0.0:1787826354.229006:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332666368 00000100:00100000:0.0:1787826354.229084:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11189 00000100:00100000:0.0:1787826354.229095:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332666368:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:0.0:1787826354.229160:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.229187:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4bd6a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801988 expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.229207:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cd900) 00010000:00010000:0.0:1787826354.229213:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826354.229270:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332666368:12345-192.168.203.56@tcp:103:rm.0 Request processed in 178us (650us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.229278:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11189 00000100:00100000:3.0:1787826354.229747:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332666624 00000100:00100000:3.0:1787826354.229754:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332666624 00000100:00100000:3.0:1787826354.229801:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9386 00000100:00100000:3.0:1787826354.229811:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226253:x1874663332666624:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:3.0:1787826354.230015:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226253:x1874663332666624:12345-192.168.203.56@tcp:35:rm.0 Request processed in 206us (1331us total) trans 4294985468 rc 0/0 00000100:00100000:3.0:1787826354.230023:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9386 00000100:00100000:1.0:1787826354.235026:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826354.235163:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332666880 00000100:00100000:2.0:1787826354.235173:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332666880 00000100:00100000:2.0:1787826354.235276:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68630 00000100:00100000:2.0:1787826354.235288:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226253:x1874663332666880:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:2.0:1787826354.235361:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826354.235389:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4bedd lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e58:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ff expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.235412:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf600/0xe21309aa81f4be9e lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019e3 expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.235425:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4bd78 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080198f expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.235435:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:2.0:1787826354.235440:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826354.235486:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.235496:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.235506:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.235529:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346312320:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1787826354.235580:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.235681:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.235741:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346312320 00000100:00100000:1.0:1787826354.235746:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346312320 00000100:00100000:1.0:1787826354.235818:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24039 00000100:00100000:1.0:1787826354.235826:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:17813:x1874663346312320:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826354.235840:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.235853:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bf4d expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.235873:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.235882:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.235899:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.235924:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.235938:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 7 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.235949:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346312448:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826354.236029:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.236038:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 8 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.236049:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346312576:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826354.236072:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.250026:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346312448:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826354.256832:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.256936:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346312576:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826354.256984:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.256997:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826354.257000:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826354.257004:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 14 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.257015:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:3.0:1787826354.257079:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:17813:x1874663346312320:12345-0@lo:101:mdt00_005.0 Request processed in 21252us (21496us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.257086:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24039 00000100:00100000:2.0:1787826354.258392:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332667136 00000100:00100000:2.0:1787826354.258404:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332667136 00010000:00010000:2.0:1787826354.258453:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 9 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.258496:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22039 00000100:00100000:2.0:1787826354.258506:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7947dfd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332667136:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826354.258520:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826354.258527:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 9 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.258539:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 9 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.258563:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4bd4e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801981 expref: 9 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.258579:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826354.258587:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.258601:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826354.258668:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dfd450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332667136:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 163us (1845us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826354.258675:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22039 00000100:00100000:0.0:1787826354.259294:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346312320:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826354.259317:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.259327:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.260017:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826354.260086:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332667008 00000100:00100000:2.0:1787826354.260092:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332667008 00010000:00010000:2.0:1787826354.260170:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 8 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.260197:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22040 00000100:00100000:2.0:1787826354.260205:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332667008:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826354.260217:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826354.260223:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 8 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.260235:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 8 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826354.260332:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787826354.260379:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4bdb7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019ab expref: 8 pid: 24702 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.260395:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826354.260400:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.260411:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.260420:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.260450:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.260465:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346312704:0@lo:105: 00000100:00100000:2.0:1787826354.260511:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.260552:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826354.260605:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346312704 00000100:00100000:3.0:1787826354.260609:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346312704 00000100:00100000:3.0:1787826354.260655:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7910 00000100:00100000:3.0:1787826354.260661:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346312704:12345-0@lo:105: 00010000:00010000:3.0:1787826354.260671:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.260681:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.260689:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.260696:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.260704:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.260711:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.260734:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef000) 00000100:00100000:3.0:1787826354.260771:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346312704:12345-0@lo:105: Request processed in 110us (259us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.260777:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7910 00000100:00100000:3.0:1787826354.260810:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346312704:0@lo:105: 00010000:00010000:3.0:1787826354.260828:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826354.260863:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332667008:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 658us (851us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.260869:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22040 00010000:00010000:2.0:1787826354.261373:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.261386:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826354.261408:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826354.261455:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.261463:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.261472:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.261490:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346312832:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1787826354.261528:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.261557:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.261609:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346312832 00000100:00100000:1.0:1787826354.261615:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346312832 00000100:00100000:1.0:1787826354.261653:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24040 00000100:00100000:1.0:1787826354.261661:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:17813:x1874663346312832:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826354.261677:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.261692:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf62 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bf5b expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.261713:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf62 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bf5b expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.261725:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf62 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf5b expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.261733:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826354.261736:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826354.261739:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826354.261784:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:17813:x1874663346312832:12345-0@lo:101:mdt00_005.0 Request processed in 124us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.261791:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24040 00000100:00100000:3.0:1787826354.263267:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346312832:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787826354.263296:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.263310:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826354.263334:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826354.263398:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346312960:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787826354.263453:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826354.263490:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.263711:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346312960 00000100:00100000:0.0:1787826354.263718:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346312960 00000100:00100000:0.0:1787826354.263769:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24041 00000100:00100000:0.0:1787826354.263778:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:17813:x1874663346312960:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826354.263858:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:17813:x1874663346312960:12345-0@lo:1000:mdt00_005.0 Request processed in 81us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.263865:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24041 00000100:00100000:2.0:1787826354.264025:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346312960:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787826354.264074:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346313088:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787826354.264102:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.264137:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.264188:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346313088 00000100:00100000:1.0:1787826354.264192:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346313088 00000100:00100000:1.0:1787826354.264225:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24042 00000100:00100000:1.0:1787826354.264231:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:17813:x1874663346313088:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826354.264345:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:17813:x1874663346313088:12345-0@lo:1000:mdt00_005.0 Request processed in 114us (241us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826354.264364:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24042 00000100:00100000:1.0:1787826354.264392:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346313088:0@lo:1000:mdt00_005.0 00010000:00010000:1.0:1787826354.264446:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.264455:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.264465:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.264476:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346313216:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1787826354.264500:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.264515:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.264527:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346313216 00000100:00100000:1.0:1787826354.264530:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346313216 00000100:00100000:1.0:1787826354.264550:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24043 00000100:00100000:1.0:1787826354.264555:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:17813:x1874663346313216:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826354.264563:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.264577:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bf69 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.264598:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 6 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.264608:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bf69 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.264640:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 6 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.264652:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 6 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.264662:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346313344:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826354.264723:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.271040:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346313344:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826354.271085:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf69 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.271141:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cd900) 00010000:00010000:1.0:1787826354.271145:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826354.271150:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf69 expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.271160:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826354.271225:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+16:17813:x1874663346313216:12345-0@lo:101:mdt00_005.0 Request processed in 6669us (6723us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.271234:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24043 00000100:00100000:2.0:1787826354.271279:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346313216:0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826354.271298:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.271312:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.272175:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.272244:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332667264 00000100:00100000:0.0:1787826354.272316:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332667264 00010000:00010000:0.0:1787826354.272367:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 7 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.272409:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22041 00000100:00100000:0.0:1787826354.272419:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332667264:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826354.272433:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.272441:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 7 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.272453:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 7 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.272476:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4bdd3 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019b2 expref: 7 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.272497:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cd900) 00010000:00010000:0.0:1787826354.272503:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf69 expref: 15 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.272513:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bf69 expref: 15 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.272521:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bf69 expref: 15 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.272552:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf69 expref: 15 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.272567:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346313472:0@lo:105: 00000100:00100000:0.0:1787826354.272609:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.272660:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.272991:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346313472 00000100:00100000:1.0:1787826354.272997:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346313472 00000100:00100000:1.0:1787826354.273081:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7911 00000100:00100000:1.0:1787826354.273090:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346313472:12345-0@lo:105: 00010000:00010000:1.0:1787826354.273101:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.273147:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.273156:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.273163:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.273364:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.273373:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.273399:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf800) 00010000:00010000:2.0:1787826354.273438:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273502:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.273629:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346313472:12345-0@lo:105: Request processed in 529us (1007us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.273639:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7911 00010000:00010000:2.0:1787826354.273645:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273659:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273670:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273696:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273709:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273716:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273745:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273752:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.273762:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.273780:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5a680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346313600:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1787826354.273816:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.273843:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.273896:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346313472:0@lo:105: 00010000:00010000:0.0:1787826354.273911:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826354.273952:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332667264:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1533us (1781us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.273960:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22041 00000100:00100000:1.0:1787826354.275586:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346313600 00000100:00100000:1.0:1787826354.275601:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346313600 00000100:00100000:1.0:1787826354.275653:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24044 00000100:00100000:1.0:1787826354.275663:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+16:17813:x1874663346313600:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826354.275685:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.275706:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 2/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bf85 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.275738:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.275748:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 4/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 17 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.275782:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.275796:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 7 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.275815:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346313728:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826354.275904:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.284366:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346313728:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826354.284409:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 3/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 17 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.284422:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:1.0:1787826354.284426:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826354.284430:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 3/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 17 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.284441:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 3/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 17 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.284449:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:2.0:1787826354.285639:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.285850:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+17:17813:x1874663346313600:12345-0@lo:101:mdt00_005.0 Request processed in 10184us (12028us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.285861:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24044 00000100:00100000:3.0:1787826354.285899:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5a680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346313600:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787826354.285931:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.285945:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.288325:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332667392 00000100:00100000:0.0:1787826354.288340:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332667392 00010000:00010000:0.0:1787826354.288400:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 8 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.288564:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22042 00000100:00100000:0.0:1787826354.288575:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332667392:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826354.288591:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.288599:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 8 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.288612:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 8 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.288635:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4beba lrc: 0/0,0 mode: --/PR res: [0x200000401:0x198:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e87608019f1 expref: 8 pid: 24703 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.288652:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:0.0:1787826354.288660:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 2/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.288672:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.288682:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.288716:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.288733:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346313856:0@lo:105: 00000100:00100000:0.0:1787826354.288797:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.288845:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826354.288848:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346313856 00000100:00100000:3.0:1787826354.288855:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346313856 00000100:00100000:3.0:1787826354.288949:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7912 00000100:00100000:3.0:1787826354.288958:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346313856:12345-0@lo:105: 00010000:00010000:3.0:1787826354.288969:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.288981:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.288990:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.288998:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.289004:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.289012:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.289042:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ef800) 00000100:00100000:3.0:1787826354.289091:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346313856:12345-0@lo:105: Request processed in 135us (294us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.289099:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7912 00000100:00100000:1.0:1787826354.289196:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346313856:0@lo:105: 00010000:00010000:1.0:1787826354.289215:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826354.289262:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.289266:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332667392:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 692us (3635us total) trans 0 rc 0/0 00010000:00010000:2.0:1787826354.289283:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.289284:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22042 00000100:00100000:2.0:1787826354.290238:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5a680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346313984:0@lo:403:mdt00_005.0 00000100:00100000:2.0:1787826354.290299:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.290329:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826354.290357:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346313984 00000100:00100000:3.0:1787826354.290364:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346313984 00000100:00100000:3.0:1787826354.290407:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24045 00000100:00100000:3.0:1787826354.290416:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:17813:x1874663346313984:12345-0@lo:403:mdt00_005.0 00000100:00100000:2.0:1787826354.296389:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:17813:x1874663346313984:12345-0@lo:403:mdt00_005.0 Request processed in 5965us (6080us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826354.296404:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24045 00000100:00100000:3.0:1787826354.296672:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5a680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346313984:0@lo:403:mdt00_005.0 00000040:00080000:2.0:1787826354.298705:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1226, 952 off1118816 00000040:00080000:2.0:1787826354.299021:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1544, 1032 off1562392 00000004:00080000:2.0:1787826354.299029:0:17813:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784631a180 with version 6882 00000004:00080000:0.0:1787826354.299200:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6882 rpc_version 6882 00000020:00100000:0.0:1787826354.299239:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826354.299252:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7842c92d80 x1874663346314240/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826354.299278:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346314240:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826354.299348:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.299420:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826354.300246:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346314240 00000100:00100000:2.0:1787826354.300734:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346314240 00000100:00100000:2.0:1787826354.300874:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24046 00000100:00100000:2.0:1787826354.300886:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+17:7471:x1874663346314240:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826354.301953:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+18:7471:x1874663346314240:12345-0@lo:1000:osp_up0-1.0 Request processed in 1064us (2599us total) trans 8589957469 rc 0/0 00000100:00100000:2.0:1787826354.301965:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24046 00000100:00100000:0.0:1787826354.302227:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346314240:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787826354.303854:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e58:0x0]: returning size 4096 00010000:00010000:2.0:1787826354.303887:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.303955:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.303962:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.303968:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826354.303972:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.303979:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.303996:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826354.303999:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826354.304003:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 2/0,0 mode: --/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304009:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4bf7e lrc: 0/0,0 mode: --/EX res: [0x240000400:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304034:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304043:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304049:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304076:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826354.304089:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x198:0x0] 00010000:00010000:2.0:1787826354.304099:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304132:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304138:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304143:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826354.304146:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304152:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304163:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:2.0:1787826354.304165:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826354.304168:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304173:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4bf77 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304185:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304193:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304200:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304228:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826354.304240:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826354.304244:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304251:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304258:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304280:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:2.0:1787826354.304285:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826354.304290:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.304297:0:17813:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.304400:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977e0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226253:x1874663332666880:12345-192.168.203.56@tcp:36:rm.0 Request processed in 69110us (69380us total) trans 4294985469 rc 0/0 00000100:00100000:2.0:1787826354.304409:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68630 00010000:00010000:1.0:1787826354.304784:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787826354.304792:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304800:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826354.304805:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826354.304813:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304847:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304852:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826354.304872:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304879:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef800/0xe21309aa81f4bf85 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf8c expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304895:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787826354.304899:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304905:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826354.304908:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826354.304915:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304929:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304935:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826354.304940:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304947:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4bf69 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf70 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304958:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826354.304961:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304967:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826354.304970:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826354.304977:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304988:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.304993:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826354.305006:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.305017:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4bf4d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf54 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.305025:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826354.305026:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826354.305028:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826354.305043:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f687100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346314496:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826354.305099:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.305152:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826354.305159:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346314368:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826354.305183:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.305206:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826354.305213:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346314624:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826354.305239:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.305266:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346314496 00000100:00100000:1.0:1787826354.305272:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346314496 00000100:00100000:1.0:1787826354.305313:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346314368 00000100:00100000:1.0:1787826354.305317:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346314368 00000100:00100000:1.0:1787826354.305331:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346314624 00000100:00100000:1.0:1787826354.305333:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346314624 00000100:00100000:1.0:1787826354.305347:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11190 00000100:00100000:1.0:1787826354.305364:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+20:3655:x1874663346314496:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826354.305376:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826354.305394:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4bf70 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e58:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bf69 expref: 20 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.305407:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f79785cd900) 00010000:00010000:1.0:1787826354.305415:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826354.305441:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+19:3655:x1874663346314496:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (342us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.305447:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11190 00000100:00100000:1.0:1787826354.305462:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11191 00000100:00100000:1.0:1787826354.305467:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+18:3655:x1874663346314368:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826354.305474:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826354.305483:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4bf8c lrc: 0/0,0 mode: --/EX res: [0x200000401:0x198:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bf85 expref: 18 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.305492:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x198:0x0].0x0 (ffff9f79785ccc00) 00010000:00010000:1.0:1787826354.305497:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826354.305514:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+17:3655:x1874663346314368:12345-0@lo:103:ldlm_bl.0 Request processed in 47us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.305519:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11191 00000100:00100000:1.0:1787826354.305528:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11192 00000100:00100000:1.0:1787826354.305533:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+16:3653:x1874663346314624:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826354.305540:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826354.305549:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4bf54 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bf4d expref: 16 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.305557:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826354.305561:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826354.305579:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+15:3653:x1874663346314624:12345-0@lo:103:ldlm_bl.0 Request processed in 46us (340us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.305584:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11192 00000100:00100000:1.0:1787826354.305613:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346314368:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826354.305621:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f687100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346314496:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826354.305656:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346314624:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826354.306369:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826354.306566:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826354.349079:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826354.404163:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.404280:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332667520 00000100:00100000:3.0:1787826354.404290:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332667520 00000100:00100000:3.0:1787826354.404367:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68631 00000100:00100000:3.0:1787826354.404379:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226261:x1874663332667520:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:3.0:1787826354.404414:0:35094:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:3.0:1787826354.404485:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226261:x1874663332667520:12345-192.168.203.56@tcp:41:stat.0 Request processed in 108us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.404492:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68631 00000100:00100000:3.0:1787826354.448371:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.450492:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332667648 00000100:00100000:0.0:1787826354.450512:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332667648 00000100:00100000:0.0:1787826354.450604:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68632 00000100:00100000:0.0:1787826354.450624:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226266:x1874663332667648:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826354.450646:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826354.450671:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bf93 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a14 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.450747:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.450761:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.450772:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826354.450827:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826354.450831:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826354.450866:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.450895:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4bf93 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.450914:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.450923:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826354.450926:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826354.450930:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826354.451246:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226266:x1874663332667648:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 630us (2894us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.451254:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68632 00000100:00100000:0.0:1787826354.460537:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826354.461156:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332667776 00000100:00100000:2.0:1787826354.461167:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332667776 00000100:00100000:2.0:1787826354.461237:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68633 00000100:00100000:2.0:1787826354.461249:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226266:x1874663332667776:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:2.0:1787826354.461272:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826354.461293:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4bfa1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a1b expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.461374:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.461389:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.461399:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.461411:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.461423:0:6527:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf62 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bf5b expref: 14 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.461432:0:6527:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.461469:0:6527:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf62 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4bf5b expref: 14 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.461487:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb1500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346314752:0@lo:104: 00000100:00100000:2.0:1787826354.461539:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.461585:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.461804:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346314752 00000100:00100000:1.0:1787826354.461809:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346314752 00000100:00100000:1.0:1787826354.461860:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7913 00000100:00100000:1.0:1787826354.461869:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346314752:12345-0@lo:104: 00010000:00010000:1.0:1787826354.461883:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.461952:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826354.461958:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346314752:12345-0@lo:104: Request processed in 91us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.461965:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7913 00010000:00010000:3.0:1787826354.462026:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826354.462033:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.462042:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efe00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826354.462048:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826354.462056:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.462085:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.462091:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826354.462141:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.462149:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490efe00/0xe21309aa81f4bf5b lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bf62 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.462174:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826354.462176:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826354.462178:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826354.462186:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826354.462197:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850359880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346314880:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826354.462236:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826354.462273:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346314880 00000100:00100000:3.0:1787826354.462277:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346314880 00010000:00010000:3.0:1787826354.462298:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf62 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4bf5b expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.462322:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22043 00000100:00100000:3.0:1787826354.462330:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c82450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+15:3652:x1874663346314880:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826354.462340:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787826354.463690:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb1500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346314752:0@lo:104: 00010000:00010000:2.0:1787826354.463719:0:6527:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.467344:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bf2a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826354.467377:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bf2a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.467429:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf2a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.467436:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826354.467476:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43c00/0xe21309aa81f4bf23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bf2a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826354.467499:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957469 is committed 00000001:00080000:1.0:1787826354.467510:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957468 is committed 00000004:00080000:1.0:1787826354.467516:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 215 committed 00010000:00010000:3.0:1787826354.467903:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4bf62 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4bf5b expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.467921:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1d00) 00010000:00010000:3.0:1787826354.467927:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.467937:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.467946:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.467980:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826354.468062:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c82450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346314880:12345-0@lo:103:ldlm_bl.0 Request processed in 5732us (5824us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.468073:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22043 00000100:00100000:3.0:1787826354.468202:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850359880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346314880:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826354.468290:0:6527:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468299:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468365:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468374:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468381:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.468382:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850772a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346315008:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826354.468395:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468401:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826354.468406:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468413:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468424:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826354.468427:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826354.468431:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.468438:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826354.468446:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4bfa8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468460:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468467:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468473:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.468474:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:1.0:1787826354.468483:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00000100:00100000:0.0:1787826354.468484:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346315136:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1787826354.468488:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468497:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468506:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826354.468508:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826354.468511:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.468518:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826354.468524:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4bfaf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.468552:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346315136 00000100:00100000:0.0:1787826354.468559:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346315136 00010000:00010000:1.0:1787826354.468570:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4bfa1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a1b expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468582:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826354.468585:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826354.468590:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee200/0xe21309aa81f4bfa1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a1b expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.468599:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:0.0:1787826354.468613:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30875 00000100:00100000:0.0:1787826354.468625:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346315136:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1787826354.468650:0:20167:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1314 count 1 00010000:00010000:0.0:1787826354.468675:0:20167:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.468692:0:20167:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.468701:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x522:0x0].0x0 since=7s v=(0 0) 00010000:00010000:0.0:1787826354.468706:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826354.468710:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.468738:0:20167:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 5 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.468751:0:20167:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 5 pid: 35352 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.468763:0:20167:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346315264:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826354.468812:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226266:x1874663332667776:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 7564us (8284us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826354.468820:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68633 00000100:00080000:2.0:1787826354.468851:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826354.468869:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346315008 00000100:00100000:2.0:1787826354.468874:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346315008 00000100:00100000:2.0:1787826354.468920:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11193 00000100:00100000:2.0:1787826354.468928:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346315008:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787826354.468939:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826354.468956:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eea00/0xe21309aa81f4bf2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bf23 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.468969:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19a:0x0].0x5a734ce9 (ffff9f7967ac8600) 00010000:00010000:2.0:1787826354.468976:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826354.472042:0:20167:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826354.474315:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346315008:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 5377us (5869us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826354.474332:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11193 00000100:00100000:0.0:1787826354.477195:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850772a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346315008:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787826354.480335:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.480516:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332667904 00000100:00100000:0.0:1787826354.480525:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332667904 00000100:00100000:0.0:1787826354.480586:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68634 00000100:00100000:0.0:1787826354.480597:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226266:x1874663332667904:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826354.480653:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826354.480675:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfbd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a22 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.480751:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.480764:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.480776:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826354.480819:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826354.480823:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826354.480842:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.480868:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfbd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.480884:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.480892:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826354.480895:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826354.480899:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826354.488267:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850359880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226266:x1874663332667904:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 7657us (7927us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.488281:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68634 00000100:00100000:0.0:1787826354.491385:0:20167:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346315264:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826354.491421:0:20167:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.491891:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.491952:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332668032 00000100:00100000:0.0:1787826354.491959:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332668032 00010000:00010000:0.0:1787826354.492014:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 6 pid: 35352 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.492051:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22044 00000100:00100000:0.0:1787826354.492061:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332668032:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826354.492076:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.492082:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 6 pid: 35352 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492095:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 6 pid: 35352 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492150:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a57c3a21/0xe21309aa81f4bd39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x522:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e876080197a expref: 6 pid: 35352 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492167:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826354.492171:0:6513:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x522:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826354.492176:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492190:0:6513:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492214:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00080000:2.0:1787826354.492298:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826354.492628:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332668032:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 567us (736us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.492636:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22044 00010000:00010000:0.0:1787826354.492668:0:20167:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492677:0:20167:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492688:0:20167:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492694:0:20167:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.492715:0:20167:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826354.492909:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346315136:12345-0@lo:6:osp-syn-0-0.0 Request processed in 24286us (24392us total) trans 4294973860 rc 0/0 00000100:00100000:0.0:1787826354.492916:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30875 00000004:00080000:0.0:1787826354.492956:0:3654:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f785035ad80/2, rc 0, transno 4294973860 00000100:00100000:0.0:1787826354.492965:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346315136:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1787826354.499047:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826354.499063:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.499078:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826354.499083:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.499094:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.503033:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.503053:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039f02c1e/0xe21309aa81f4bfb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x522:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.503084:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826354.503087:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826354.503090:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826354.503197:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.503302:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332668288 00000100:00100000:0.0:1787826354.503312:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332668288 00000100:00100000:0.0:1787826354.503402:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68635 00000100:00100000:0.0:1787826354.503414:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226266:x1874663332668288:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:0.0:1787826354.503547:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346315392:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.503602:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.503645:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.503667:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346315392 00000100:00100000:0.0:1787826354.503671:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346315392 00000100:00100000:0.0:1787826354.503708:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24047 00000100:00100000:0.0:1787826354.503715:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:35095:x1874663346315392:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.503954:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:35095:x1874663346315392:12345-0@lo:1000:mdt00_012.0 Request processed in 239us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.503962:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24047 00000100:00100000:0.0:1787826354.504000:0:35095:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957469 gen 2 00000004:00080000:0.0:1787826354.504005:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957462 committed transno 8589957469 00000020:00080000:0.0:1787826354.504011:0:35095:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784d00ed80 refcount 2 committed 0 result 0 batchid 3438 00000020:00080000:0.0:1787826354.504015:0:35095:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed35300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d00e9c0 00000020:00080000:0.0:1787826354.504018:0:35095:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1542 00000020:00080000:0.0:1787826354.504021:0:35095:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784421e700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784e775400 00000020:00080000:0.0:1787826354.504023:0:35095:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1224 00000004:00080000:0.0:1787826354.504047:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957463 committed transno 8589957469 00000004:00080000:0.0:1787826354.504059:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957464 committed transno 8589957469 00000004:00080000:0.0:1787826354.504072:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957465 committed transno 8589957469 00000004:00080000:0.0:1787826354.504081:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957466 committed transno 8589957469 00000004:00080000:0.0:1787826354.504090:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957467 committed transno 8589957469 00000004:00080000:0.0:1787826354.504133:0:35095:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957469 committed transno 8589957469 00000100:00100000:0.0:1787826354.504149:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346315392:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.504191:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346315520:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.504217:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.504235:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a180 going to sleep for 11 seconds 00000020:00080000:0.0:1787826354.504246:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3438 committed batchid 3437 00000020:00080000:0.0:1787826354.504307:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3437 00000100:00100000:0.0:1787826354.504319:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346315520 00000100:00100000:0.0:1787826354.504323:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346315520 00000100:00100000:0.0:1787826354.504362:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24048 00000100:00100000:0.0:1787826354.504369:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:35095:x1874663346315520:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.504450:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:35095:x1874663346315520:12345-0@lo:1000:mdt00_012.0 Request processed in 82us (232us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826354.504457:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24048 00000100:00100000:0.0:1787826354.504484:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346315520:0@lo:1000:mdt00_012.0 00010000:00010000:0.0:1787826354.504528:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.504538:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.504547:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.504558:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346315648:0@lo:101:mdt00_012.0 00000100:00100000:0.0:1787826354.504583:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.504597:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.504608:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346315648 00000100:00100000:0.0:1787826354.504611:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346315648 00000100:00100000:0.0:1787826354.504933:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24049 00000100:00100000:0.0:1787826354.504940:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:35095:x1874663346315648:12345-0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826354.504949:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826354.504961:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bfcb expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.504987:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.504996:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.505004:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.505028:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 6 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.505041:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 6 pid: 35080 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.505052:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346315776:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826354.505143:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.505153:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 6 pid: 6528 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.505168:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346315904:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826354.505193:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb73c0 going to sleep for 11 seconds 00000100:00080000:1.0:1787826354.505276:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826354.514804:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.514964:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346315776:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826354.514978:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346315904:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826354.515022:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.515035:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826354.515039:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826354.515043:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.515054:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826354.515138:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346315648:12345-0@lo:101:mdt00_012.0 Request processed in 10198us (10553us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.515146:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24049 00000100:00100000:2.0:1787826354.515271:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a91f480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346315648:0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826354.515289:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.515300:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.515568:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826354.527715:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332668544 00000100:00100000:3.0:1787826354.527844:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332668416 00000100:00100000:3.0:1787826354.527858:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332668416 00010000:00010000:3.0:1787826354.527955:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 7 pid: 35080 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.528291:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22045 00000100:00100000:3.0:1787826354.528304:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332668416:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826354.528321:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826354.528329:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 7 pid: 35080 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.528345:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 7 pid: 35080 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.529072:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4bfc4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a22 expref: 7 pid: 35080 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.529092:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826354.529098:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.529130:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826354.529261:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332668416:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 959us (14466us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.529269:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22045 00000100:00100000:2.0:1787826354.529346:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332668544 00010000:00010000:0.0:1787826354.535592:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 7 pid: 6528 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.535653:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22046 00000100:00100000:0.0:1787826354.535666:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332668544:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826354.535680:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.535688:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 7 pid: 6528 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.535705:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 7 pid: 6528 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.535736:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4bf9a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a14 expref: 7 pid: 6528 timeout: 8266 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.535753:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826354.535760:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.535771:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.535780:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.535819:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826354.535834:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346316032:0@lo:105: 00000100:00100000:0.0:1787826354.535896:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.535939:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb73c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.538361:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346316032 00000100:00100000:1.0:1787826354.538374:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346316032 00000100:00100000:1.0:1787826354.538449:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7914 00000100:00100000:1.0:1787826354.538460:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346316032:12345-0@lo:105: 00010000:00010000:1.0:1787826354.538477:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538488:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538498:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538506:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538514:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538529:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538558:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43a00) 00000100:00100000:1.0:1787826354.538648:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346316032:12345-0@lo:105: Request processed in 190us (2751us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.538655:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7914 00010000:00010000:1.0:1787826354.538685:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538694:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826354.538738:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826354.538779:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538788:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.538799:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.538818:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850773480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316160:0@lo:101:mdt00_012.0 00000100:00100000:1.0:1787826354.538854:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.538876:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.539064:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346316032:0@lo:105: 00010000:00010000:0.0:1787826354.539081:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826354.539155:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332668544:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3490us (23585us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.539168:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22046 00000100:00100000:0.0:1787826354.539197:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346316160 00000100:00100000:0.0:1787826354.539202:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346316160 00000100:00100000:0.0:1787826354.539231:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24050 00000100:00100000:0.0:1787826354.539239:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:35095:x1874663346316160:12345-0@lo:101:mdt00_012.0 00010000:00010000:0.0:1787826354.539254:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826354.539270:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfe0 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bfd9 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.539293:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfe0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bfd9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.539306:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfe0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bfd9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.539314:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57ac00) 00010000:00010000:0.0:1787826354.539317:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826354.539321:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826354.539368:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316160:12345-0@lo:101:mdt00_012.0 Request processed in 130us (513us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.539373:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24050 00000100:00100000:1.0:1787826354.539861:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850773480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316160:0@lo:101:mdt00_012.0 00010000:00010000:1.0:1787826354.539880:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.539889:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826354.539908:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787826354.539949:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850773480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316288:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826354.539983:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.540009:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826354.540422:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346316288 00000100:00100000:3.0:1787826354.540429:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346316288 00000100:00100000:3.0:1787826354.540471:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24051 00000100:00100000:3.0:1787826354.540481:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316288:12345-0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826354.540549:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316288:12345-0@lo:1000:mdt00_012.0 Request processed in 70us (565us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.540557:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24051 00000100:00100000:1.0:1787826354.540711:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850773480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316288:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826354.540923:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850773480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316416:0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826354.540954:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.540980:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.541030:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346316416 00000100:00100000:0.0:1787826354.541035:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346316416 00000100:00100000:0.0:1787826354.541073:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24052 00000100:00100000:0.0:1787826354.541082:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316416:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.541246:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316416:12345-0@lo:1000:mdt00_012.0 Request processed in 164us (290us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826354.541253:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24052 00000100:00100000:0.0:1787826354.541283:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850773480 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316416:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.541370:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316544:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.541398:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.541415:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.541427:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346316544 00000100:00100000:0.0:1787826354.541430:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346316544 00000100:00100000:0.0:1787826354.541453:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24053 00000100:00100000:0.0:1787826354.541459:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316544:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.541514:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316544:12345-0@lo:1000:mdt00_012.0 Request processed in 55us (115us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787826354.541520:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24053 00000100:00100000:0.0:1787826354.541545:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316544:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.541591:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316672:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.541614:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.542278:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.546417:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346316672 00000100:00100000:1.0:1787826354.546434:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346316672 00000100:00100000:1.0:1787826354.546540:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24054 00000100:00100000:1.0:1787826354.546551:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316672:12345-0@lo:1000:mdt00_012.0 00000100:00100000:1.0:1787826354.546879:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316672:12345-0@lo:1000:mdt00_012.0 Request processed in 329us (5262us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.546887:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24054 00000100:00100000:2.0:1787826354.557469:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316672:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826354.557587:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb1500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316800:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826354.557658:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.557705:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.557822:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346316800 00000100:00100000:0.0:1787826354.557829:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346316800 00000100:00100000:0.0:1787826354.557880:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24055 00000100:00100000:0.0:1787826354.557890:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316800:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.557981:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316800:12345-0@lo:1000:mdt00_012.0 Request processed in 93us (324us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.557988:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24055 00000100:00100000:2.0:1787826354.558163:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb1500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316800:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826354.558216:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb1500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316928:0@lo:1000:mdt00_012.0 00000100:00100000:2.0:1787826354.558240:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.558264:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.558381:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346316928 00000100:00100000:0.0:1787826354.558385:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346316928 00000100:00100000:0.0:1787826354.558419:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24056 00000100:00100000:0.0:1787826354.558425:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316928:12345-0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826354.558504:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35095:x1874663346316928:12345-0@lo:1000:mdt00_012.0 Request processed in 69us (252us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.558511:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24056 00000100:00100000:2.0:1787826354.558636:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb1500 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346316928:0@lo:1000:mdt00_012.0 00000040:00080000:2.0:1787826354.559462:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1227, 2168 off1120984 00000040:00080000:2.0:1787826354.560417:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1545, 2368 off1564760 00000004:00080000:2.0:1787826354.560426:0:35095:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7850705d80 with version 6883 00000004:00080000:0.0:1787826354.560487:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6883 rpc_version 6883 00000020:00100000:0.0:1787826354.560527:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826354.560539:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977f29c00 x1874663346317184/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826354.560562:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346317184:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826354.560631:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.560704:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.562990:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346317184 00000100:00100000:1.0:1787826354.563004:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346317184 00000100:00100000:0.0:1787826354.567045:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24057 00000100:00100000:0.0:1787826354.567061:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346317184:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826354.567947:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346317184:12345-0@lo:1000:osp_up0-1.0 Request processed in 887us (7313us total) trans 8589957470 rc 0/0 00000100:00100000:0.0:1787826354.567956:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24057 00000100:00100000:2.0:1787826354.568125:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346317184:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826354.568248:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.568263:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.568274:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.568307:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.568314:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.568321:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.568363:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.568372:0:35095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.568380:0:35095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.568401:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009f80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346317312:0@lo:101:mdt00_012.0 00000100:00100000:2.0:1787826354.568438:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.571648:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346317312 00000100:00100000:1.0:1787826354.571663:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346317312 00000100:00100000:1.0:1787826354.571743:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24058 00000100:00100000:1.0:1787826354.571752:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35095:x1874663346317312:12345-0@lo:101:mdt00_012.0 00010000:00010000:1.0:1787826354.571778:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.571801:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bffc lrc: 2/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4bff5 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.571831:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bffc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4bff5 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.571844:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bffc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bff5 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.571852:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:1.0:1787826354.571855:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826354.571858:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826354.571916:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35095:x1874663346317312:12345-0@lo:101:mdt00_012.0 Request processed in 166us (3476us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.571922:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24058 00000100:00100000:2.0:1787826354.575954:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826354.576016:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009f80 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-mdtlov_UUID:35095:1874663346317312:0@lo:101:mdt00_012.0 00010000:00010000:2.0:1787826354.576042:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576057:0:35095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826354.576089:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001cde80ad reply state 00000000f02a2d88 transno 4294985470 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576102:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576129:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826354.576133:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826354.576137:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826354.576144:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576153:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826354.576165:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x199:0x0] 00000004:00010000:2.0:1787826354.576174:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001cde80ad reply state 00000000f02a2d88 transno 4294985470 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576182:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576188:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:2.0:1787826354.576190:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826354.576192:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787826354.576198:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5a:0x0]: returning size 4096 00000004:00080000:2.0:1787826354.576210:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826354.576219:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576226:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576233:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576263:0:35095:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787826354.576267:0:35095:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826354.576271:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.576278:0:35095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.576394:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226266:x1874663332668288:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 72979us (73197us total) trans 4294985470 rc 0/0 00000100:00100000:2.0:1787826354.576402:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68635 00010000:00010000:0.0:1787826354.576487:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826354.576495:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.576503:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826354.576509:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826354.576517:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.576559:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.576565:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826354.576600:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.576607:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43a00/0xe21309aa81f4bfcb lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4bfd2 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.576621:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826354.576623:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826354.576625:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826354.576632:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826354.576646:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346317440:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826354.576703:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826354.576757:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346317440 00000100:00100000:0.0:1787826354.576764:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346317440 00000100:00100000:0.0:1787826354.576804:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11194 00000100:00100000:0.0:1787826354.576812:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346317440:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826354.576824:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826354.576843:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4bfd2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4bfcb expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.576855:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826354.576859:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826354.576886:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346317440:12345-0@lo:103:ldlm_bl.0 Request processed in 75us (183us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.576892:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11194 00000100:00100000:0.0:1787826354.576920:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346317440:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826354.587680:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.587852:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332668672 00000100:00100000:1.0:1787826354.587863:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332668672 00000100:00100000:1.0:1787826354.587931:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68636 00000100:00100000:1.0:1787826354.587942:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226254:x1874663332668672:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826354.587963:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826354.587985:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c003 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a29 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.588052:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c00a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.588065:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c00a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.588078:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c00a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826354.591390:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826354.591405:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826354.591446:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c00a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.591494:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c003 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a29 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.591519:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c00a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a29 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.591528:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826354.591531:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826354.591535:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:0.0:1787826354.591611:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226254:x1874663332668672:12345-192.168.203.56@tcp:101:bash.0 Request processed in 3670us (3940us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.591620:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68636 00000100:00100000:1.0:1787826354.602150:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.602234:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332668800 00000100:00100000:3.0:1787826354.602244:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332668800 00000100:00100000:3.0:1787826354.602305:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68637 00000100:00100000:3.0:1787826354.602316:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226254:x1874663332668800:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826354.602337:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.602367:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c011 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a30 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.602436:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.602451:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.602463:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.602474:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.602484:0:24703:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfe0 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bfd9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.602491:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.602529:0:24703:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfe0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4bfd9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.602548:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346317568:0@lo:104: 00000100:00100000:3.0:1787826354.602599:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826354.602634:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.602928:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346317568 00000100:00100000:0.0:1787826354.602934:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346317568 00000100:00100000:0.0:1787826354.602975:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7915 00000100:00100000:0.0:1787826354.602984:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346317568:12345-0@lo:104: 00010000:00010000:0.0:1787826354.602997:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.603141:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346317568:0@lo:104: 00010000:00010000:3.0:1787826354.603163:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.603225:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826354.603232:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346317568:12345-0@lo:104: Request processed in 249us (632us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.603239:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7915 00010000:00010000:2.0:1787826354.605676:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826354.605691:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.605704:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826354.605711:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826354.605798:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.605922:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.605930:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826354.605966:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.605974:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece800/0xe21309aa81f4bfd9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bfe0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.606000:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826354.606002:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826354.606004:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826354.606045:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826354.606070:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346317696:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826354.606168:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826354.606269:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346317696 00000100:00100000:1.0:1787826354.606278:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346317696 00010000:00010000:1.0:1787826354.606311:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfe0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4bfd9 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826354.606357:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22047 00000100:00100000:1.0:1787826354.606367:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346317696:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826354.606380:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1787826354.619440:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957470 is committed 00010000:00010000:1.0:1787826354.619609:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4bfe0 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4bfd9 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.619643:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57ac00) 00010000:00010000:1.0:1787826354.619649:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.619660:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.619668:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.619699:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826354.619795:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346317696:12345-0@lo:103:ldlm_bl.0 Request processed in 13428us (13626us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.619810:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22047 00000100:00100000:2.0:1787826354.623554:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346317696:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826354.623740:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.623751:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.623908:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346317824:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826354.623979:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826354.624012:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826354.624144:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346317824 00000100:00100000:1.0:1787826354.624151:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346317824 00000100:00100000:1.0:1787826354.624197:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24059 00000100:00100000:1.0:1787826354.624206:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:24703:x1874663346317824:12345-0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826354.624283:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:24703:x1874663346317824:12345-0@lo:1000:mdt00_007.0 Request processed in 77us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.624290:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24059 00000100:00100000:3.0:1787826354.624591:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-mdtlov_UUID:24703:1874663346317824:0@lo:1000:mdt00_007.0 00010000:00010000:3.0:1787826354.624701:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c026 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624716:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c026 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624728:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c026 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826354.624751:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c026 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624766:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624773:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624779:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624786:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624791:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826354.624795:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624802:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624815:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826354.624819:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826354.624825:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624831:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c018 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624844:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624850:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624856:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624860:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826354.624863:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624870:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624877:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57ac00) 00010000:00010000:3.0:1787826354.624879:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826354.624882:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624889:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c01f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624929:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c011 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a30 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624944:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c026 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a30 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.624957:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826354.624960:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826354.624962:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826354.625025:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226254:x1874663332668800:12345-192.168.203.56@tcp:101:bash.0 Request processed in 22711us (22926us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.625032:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68637 00000100:00100000:1.0:1787826354.638382:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.638472:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332668928 00000100:00100000:3.0:1787826354.638484:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332668928 00000100:00100000:3.0:1787826354.638553:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68638 00000100:00100000:3.0:1787826354.638563:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226254:x1874663332668928:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826354.638592:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.638613:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c02d lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a37 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.638690:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4c034 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.638705:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4c034 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.638716:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4c034 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826354.638827:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5a:0x0]: returning size 4096 00000004:00200000:3.0:1787826354.638832:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826354.638872:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c02d lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a37 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.638895:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4c034 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a37 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.638903:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:3.0:1787826354.638906:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1787826354.638910:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826354.638975:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226254:x1874663332668928:12345-192.168.203.56@tcp:101:bash.0 Request processed in 413us (604us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.638983:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68638 00000100:00100000:1.0:1787826354.645334:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.645427:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332669056 00000100:00100000:3.0:1787826354.645437:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332669056 00000100:00100000:3.0:1787826354.645500:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68639 00000100:00100000:3.0:1787826354.645511:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226254:x1874663332669056:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826354.645531:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.645551:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c03b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a3e expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.645614:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c042 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.645639:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c042 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.645650:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c042 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826354.645701:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19b:0x0]: returning size 4096 00000004:00200000:3.0:1787826354.645707:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826354.645757:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c03b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a3e expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.645774:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c042 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a3e expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.645782:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:3.0:1787826354.645786:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826354.645790:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826354.645858:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226254:x1874663332669056:12345-192.168.203.56@tcp:101:bash.0 Request processed in 348us (534us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.645865:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68639 00000100:00100000:1.0:1787826354.665009:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.665098:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332669184 00000100:00100000:3.0:1787826354.665139:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332669184 00000100:00100000:3.0:1787826354.665204:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68640 00000100:00100000:3.0:1787826354.665215:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226254:x1874663332669184:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826354.665234:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.665257:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c049 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a45 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.665321:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.665335:0:11629:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bffc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4bff5 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.665344:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.665392:0:11629:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bffc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4bff5 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.665408:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346317952:0@lo:104: 00000100:00100000:3.0:1787826354.665461:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826354.665495:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.665598:0:6509:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826354.665610:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346317952 00000100:00100000:0.0:1787826354.665628:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346317952 00000100:00100000:0.0:1787826354.665667:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7916 00000100:00100000:0.0:1787826354.665675:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7974bcf850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346317952:12345-0@lo:104: 00010000:00010000:0.0:1787826354.665687:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.665763:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826354.665769:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7974bcf850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346317952:12345-0@lo:104: Request processed in 96us (308us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.665776:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7916 00010000:00010000:2.0:1787826354.665923:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826354.665931:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.665940:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826354.665945:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826354.665953:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.665980:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.665986:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826354.666013:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.666020:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef200/0xe21309aa81f4bff5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4bffc expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.666039:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826354.666041:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826354.666043:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826354.666051:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826354.666062:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d009f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346318080:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826354.666097:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826354.666164:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346318080 00000100:00100000:2.0:1787826354.666169:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346318080 00010000:00010000:2.0:1787826354.666186:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bffc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4bff5 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826354.666207:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22048 00000100:00100000:2.0:1787826354.666213:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c83850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346318080:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826354.666223:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826354.666649:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346317952:0@lo:104: 00010000:00010000:1.0:1787826354.666670:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.666911:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4bffc lrc: 0/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4bff5 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.666925:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:2.0:1787826354.666930:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.666939:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.666945:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.666963:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826354.667004:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c83850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346318080:12345-0@lo:103:ldlm_bl.0 Request processed in 791us (907us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826354.667011:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22048 00000100:00100000:2.0:1787826354.667063:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d009f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346318080:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826354.667149:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.667156:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826354.667212:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x199:0x0]: returning size 4096 00000004:00200000:1.0:1787826354.667216:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826354.667259:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c049 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a45 expref: 10 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.667275:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a45 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826354.667287:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:1.0:1787826354.667290:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826354.667293:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:1.0:1787826354.667345:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226254:x1874663332669184:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2132us (2348us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826354.667363:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68640 00000100:00100000:1.0:1787826354.681984:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.682829:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332669440 00000100:00100000:3.0:1787826354.682840:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332669440 00000100:00100000:3.0:1787826354.682911:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68641 00000100:00100000:3.0:1787826354.682922:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226254:x1874663332669440:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826354.682943:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.682946:0:35092:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826354.682976:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c050 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a45 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.682996:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1e00) 00010000:00010000:3.0:1787826354.683007:0:35092:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826354.683021:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c057 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a4c expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.683365:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.683377:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.683386:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.683398:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.683404:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.683422:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826354.684140:0:35092:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x523:0x0], index 1: 0 00010000:00010000:3.0:1787826354.684292:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c06c lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684304:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c06c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684312:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c06c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684345:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684364:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684370:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684377:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684387:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826354.684391:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684398:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684408:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f79785cc900) 00010000:00010000:3.0:1787826354.684411:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826354.684415:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 3/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684427:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c05e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826354.684439:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e14ebede reply state 000000008cce660c transno 8589957471 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684450:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684456:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785cc600) 00010000:00010000:3.0:1787826354.684458:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826354.684460:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826354.684498:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c057 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a4c expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684518:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c06c lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a4c expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.684530:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826354.684532:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826354.684534:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826354.685544:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226254:x1874663332669440:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2618us (3564us total) trans 8589957471 rc 0/0 00000100:00100000:3.0:1787826354.685554:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68641 00000100:00100000:0.0:1787826354.717509:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826354.717601:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332669568 00000100:00100000:2.0:1787826354.717611:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332669568 00000100:00100000:2.0:1787826354.717706:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30876 00000100:00100000:2.0:1787826354.717716:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226254:x1874663332669568:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826354.717747:0:8443:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826354.717772:0:8443:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a53 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826354.717802:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x523:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826354.717825:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=8166s v=(0 0) 00010000:00010000:2.0:1787826354.717829:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826354.717840:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 8 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.717855:0:8443:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 8 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826354.717867:0:8443:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00080000:1.0:1787826354.717999:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826354.718205:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226254:x1874663332669568:12345-192.168.203.56@tcp:101:bash.0 Request processed in 488us (705us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826354.718212:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30876 00000100:00100000:1.0:1787826354.727643:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.729026:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332669824 00000100:00100000:0.0:1787826354.729040:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332669824 00000100:00100000:0.0:1787826354.729150:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9387 00000100:00100000:0.0:1787826354.729161:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226254:x1874663332669824:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826354.729486:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226254:x1874663332669824:12345-192.168.203.56@tcp:35:bash.0 Request processed in 325us (1861us total) trans 8589957472 rc 0/0 00000100:00100000:0.0:1787826354.729494:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9387 00000100:00100000:1.0:1787826354.735970:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.736097:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332670080 00000100:00100000:3.0:1787826354.736139:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332670080 00000100:00100000:3.0:1787826354.736204:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68642 00000100:00100000:3.0:1787826354.736215:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226254:x1874663332670080:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826354.736236:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826354.736239:0:35098:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1787826354.736268:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c042 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a3e expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.736286:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:3.0:1787826354.736292:0:35098:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826354.736305:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4c081 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a5a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.736451:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.736465:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.736475:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.736491:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.736497:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.736504:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826354.736818:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035a300 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346318208:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826354.736889:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826354.736914:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826354.737065:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346318208 00000100:00100000:0.0:1787826354.737070:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346318208 00000100:00100000:0.0:1787826354.737133:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24060 00000100:00100000:0.0:1787826354.737141:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf89850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:35098:x1874663346318208:12345-0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826354.737208:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf89850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:35098:x1874663346318208:12345-0@lo:1000:mdt00_013.0 Request processed in 68us (319us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.737215:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24060 00000100:00100000:3.0:1787826354.737586:0:35098:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957470 gen 2 00000004:00080000:3.0:1787826354.737592:0:35098:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957470 committed transno 8589957470 00000100:00100000:3.0:1787826354.737976:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035a300 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346318208:0@lo:1000:mdt00_013.0 00000004:00080000:3.0:1787826354.738244:0:35098:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4d0:0x0], index 0: 0 00010000:00010000:3.0:1787826354.738401:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c096 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.738414:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c096 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.738425:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c096 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.739586:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.739600:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.739608:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.739820:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.739827:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826354.739832:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.739839:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.739856:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:3.0:1787826354.739860:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826354.740836:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.740851:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c088 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826354.740945:0:35098:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000018d8349e reply state 0000000050c2aac3 transno 4294985471 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.740960:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.740968:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x4ec94fd6 (ffff9f79785cc900) 00010000:00010000:3.0:1787826354.740970:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826354.740974:0:35098:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826354.741026:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68600/0xe21309aa81f4c081 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a5a expref: 12 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.741042:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c096 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a5a expref: 11 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.741050:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:3.0:1787826354.741052:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826354.741055:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826354.741474:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226254:x1874663332670080:12345-192.168.203.56@tcp:101:bash.0 Request processed in 5258us (5512us total) trans 4294985471 rc 0/0 00000100:00100000:3.0:1787826354.741483:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68642 00000100:00100000:2.0:1787826354.750967:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826354.751059:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332670208 00000100:00100000:0.0:1787826354.751069:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332670208 00000100:00100000:0.0:1787826354.751162:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30877 00000100:00100000:0.0:1787826354.751172:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:226254:x1874663332670208:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826354.751191:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826354.751213:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a61 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826354.751239:0:24218:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4d0:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787826354.751262:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d0:0x0].0x0 since=8166s v=(0 0) 00010000:00010000:0.0:1787826354.751267:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d0:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826354.751276:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.751291:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826354.751303:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00100000:0.0:1787826354.751369:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226254:x1874663332670208:12345-192.168.203.56@tcp:101:bash.0 Request processed in 198us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826354.751385:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30877 00000100:00100000:1.0:1787826354.757195:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826354.757284:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332670464 00000100:00100000:3.0:1787826354.757294:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332670464 00000100:00100000:3.0:1787826354.757362:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9388 00000100:00100000:3.0:1787826354.757375:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226254:x1874663332670464:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826354.757716:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:226254:x1874663332670464:12345-192.168.203.56@tcp:35:bash.0 Request processed in 342us (530us total) trans 4294985472 rc 0/0 00000100:00100000:3.0:1787826354.757724:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9388 00000100:00100000:1.0:1787826354.788428:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.788470:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826354.788602:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826354.789339:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332670592 00000100:00100000:2.0:1787826354.789358:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332670592 00000100:00100000:2.0:1787826354.789631:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332670720 00000100:00100000:2.0:1787826354.789635:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332670720 00000100:00100000:0.0:1787826354.789665:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332670848 00000100:00100000:0.0:1787826354.789671:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332670848 00000100:00100000:3.0:1787826354.789723:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11195 00000100:00100000:3.0:1787826354.789739:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:6166:x1874663332670592:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826354.789755:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826354.789783:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c00a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a29 expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.789805:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826354.789819:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c06c lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a4c expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.789828:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826354.789843:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c026 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a30 expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.789851:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f79785cc000) 00010000:00010000:3.0:1787826354.789856:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826354.789924:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:6166:x1874663332670592:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 190us (1505us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.789931:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11195 00000100:00100000:3.0:1787826354.789950:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11196 00000100:00100000:3.0:1787826354.789958:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:49149:x1874663332670720:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826354.789966:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826354.789980:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c096 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a5a expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.789992:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:3.0:1787826354.790004:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69800/0xe21309aa81f4c034 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a37 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.790013:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:3.0:1787826354.790021:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4bde1 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e58:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019b9 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.790033:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e58:0x0].0x0 (ffff9f784b4a5f00) 00010000:00010000:3.0:1787826354.790039:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826354.790059:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:49149:x1874663332670720:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 102us (1589us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.790065:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11196 00000100:00100000:3.0:1787826354.790080:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11197 00000100:00100000:3.0:1787826354.790087:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:15673:x1874663332670848:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826354.790096:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826354.790135:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4be89 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e59:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e87608019dc expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826354.790144:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e59:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:3.0:1787826354.790150:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826354.790169:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:15673:x1874663332670848:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 83us (1567us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826354.790174:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11197 00010000:00010000:0.0:1787826355.372987:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826355.432736:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826355.432783:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826355.432820:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826355.432922:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332671232 00000100:00100000:1.0:1787826355.432932:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332671232 00000100:00100000:1.0:1787826355.433002:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14364 00000100:00100000:1.0:1787826355.433014:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2395:x1874663332671232:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826355.433096:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d008000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2395:x1874663332671232:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 85us (312us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826355.433142:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14364 00000100:00100000:3.0:1787826355.433653:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332670976 00000100:00100000:3.0:1787826355.433659:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332670976 00000100:00100000:3.0:1787826355.433705:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2238 00000100:00100000:3.0:1787826355.433714:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c045850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332670976:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826355.433755:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c045850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332670976:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 43us (1031us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826355.433762:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2238 00000100:00100000:2.0:1787826355.434170:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826355.434200:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826355.434336:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332671744 00000100:00100000:3.0:1787826355.434353:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332671744 00000100:00100000:3.0:1787826355.434397:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14365 00000100:00100000:3.0:1787826355.434406:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2395:x1874663332671744:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826355.434448:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851837b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2395:x1874663332671744:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 44us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826355.434456:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14365 00000100:00100000:2.0:1787826355.438371:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826355.438412:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826355.438473:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332671104 00000100:00100000:3.0:1787826355.438483:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332671104 00000100:00100000:3.0:1787826355.438567:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14366 00000100:00100000:3.0:1787826355.438578:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332671104:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826355.438672:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332671104:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 95us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826355.438680:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14366 00000100:00100000:0.0:1787826355.441477:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332671488 00000100:00100000:0.0:1787826355.441494:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332671488 00000100:00100000:0.0:1787826355.441839:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332672000 00000100:00100000:0.0:1787826355.441845:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332672000 00000100:00100000:2.0:1787826355.455452:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826355.455499:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826355.462236:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332671360 00000100:00100000:1.0:1787826355.462252:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332671360 00000100:00100000:1.0:1787826355.462363:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8871 00000100:00100000:1.0:1787826355.462377:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332671488:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826355.462474:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332671488:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 101us (29652us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826355.462481:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8871 00000100:00100000:1.0:1787826355.462504:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8872 00000100:00100000:1.0:1787826355.462512:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332672000:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826355.462538:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332672000:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (28337us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826355.462545:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8872 00000100:00100000:1.0:1787826355.462561:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8873 00000100:00100000:1.0:1787826355.462570:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332671360:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826355.462594:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332671360:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (24182us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826355.462599:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8873 00000100:00080000:1.0:1787826355.462661:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826355.464273:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332671616 00000100:00100000:1.0:1787826355.464286:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332671616 00000100:00100000:1.0:1787826355.464489:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14367 00000100:00100000:1.0:1787826355.464502:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332671616:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826355.464830:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332671616:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 330us (9387us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826355.464838:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14367 00000100:00100000:0.0:1787826355.476081:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332671872 00000100:00100000:0.0:1787826355.476097:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332671872 00000100:00100000:0.0:1787826355.476185:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8874 00000100:00100000:0.0:1787826355.476198:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332671872:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826355.476284:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332671872:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 88us (20783us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826355.476293:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8874 00010000:00010000:0.0:1787826356.103485:0:167222:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:0.0:1787826356.103788:0:167222:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000100:00100000:3.0:1787826356.206397:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346318720:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.206475:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.206523:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346319232:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.206730:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.206753:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346319744:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.206782:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.206810:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:3.0:1787826356.206817:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346318336:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.206847:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.207136:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346318848:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.207164:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.207185:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346319360:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.207207:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.207225:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346319872:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.207258:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.207290:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346318464:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.207373:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.207391:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346318976:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.207413:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.207428:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346319488:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.207447:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.207467:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1787826356.207472:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346318592:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.207495:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.207512:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346319104:0@lo:400:kworker.0 00000100:00100000:1.0:1787826356.209322:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346318464 00000100:00100000:1.0:1787826356.209353:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346318464 00000100:00100000:1.0:1787826356.209409:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346319488 00000100:00100000:1.0:1787826356.209414:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346319488 00000100:00100000:1.0:1787826356.209444:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24061 00000100:00100000:1.0:1787826356.209455:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346318464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.211124:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.211192:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346319616:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.211249:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826356.211296:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346318464:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.211358:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346318720 00000100:00100000:3.0:1787826356.211364:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346318720 00000100:00100000:3.0:1787826356.211421:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346319232 00000100:00100000:3.0:1787826356.211424:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346319232 00000100:00100000:3.0:1787826356.211446:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346318848 00000100:00100000:3.0:1787826356.211449:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346318848 00000100:00100000:3.0:1787826356.211470:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346319872 00000100:00100000:3.0:1787826356.211473:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346319872 00000100:00100000:3.0:1787826356.211491:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346318592 00000100:00100000:3.0:1787826356.211494:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346318592 00000100:00100000:3.0:1787826356.211509:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346319104 00000100:00100000:3.0:1787826356.211512:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346319104 00000100:00100000:3.0:1787826356.211532:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14368 00000100:00100000:3.0:1787826356.211540:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346318720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.211593:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346318720:12345-0@lo:400:kworker.0 Request processed in 53us (5118us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826356.211600:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14368 00000100:00100000:3.0:1787826356.211632:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14369 00000100:00100000:3.0:1787826356.211638:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346319232:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.211667:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346319232:12345-0@lo:400:kworker.0 Request processed in 29us (4937us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826356.211672:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14369 00000100:00100000:3.0:1787826356.211684:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14370 00000100:00100000:3.0:1787826356.211689:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346318848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.211716:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346318848:12345-0@lo:400:kworker.0 Request processed in 27us (4552us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826356.211722:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14370 00000100:00100000:3.0:1787826356.211735:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14371 00000100:00100000:3.0:1787826356.211741:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346319872:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.211768:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346319872:12345-0@lo:400:kworker.0 Request processed in 28us (4520us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826356.211773:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14371 00000100:00100000:3.0:1787826356.211784:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14372 00000100:00100000:3.0:1787826356.211790:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346318592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.211818:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346318592:12345-0@lo:400:kworker.0 Request processed in 29us (4323us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826356.211825:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14372 00000100:00100000:3.0:1787826356.211836:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14373 00000100:00100000:3.0:1787826356.211842:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346319104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826356.211984:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346318464:12345-0@lo:400:kworker.0 Request processed in 2529us (4608us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826356.211994:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24061 00000100:00100000:1.0:1787826356.212018:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24062 00000100:00100000:1.0:1787826356.212026:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346319488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.212552:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346319104:12345-0@lo:400:kworker.0 Request processed in 708us (5001us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826356.212560:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14373 00000100:00100000:3.0:1787826356.212593:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346319744 00000100:00100000:3.0:1787826356.212598:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346319744 00000100:00100000:3.0:1787826356.212640:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346319360 00000100:00100000:3.0:1787826356.212643:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346319360 00000100:00100000:3.0:1787826356.212663:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346318976 00000100:00100000:3.0:1787826356.212666:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346318976 00000100:00100000:3.0:1787826356.212685:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346319616 00000100:00100000:3.0:1787826356.212688:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346319616 00000100:00100000:3.0:1787826356.212706:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8875 00000100:00100000:3.0:1787826356.212712:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346319744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826356.214138:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8876 00000100:00100000:0.0:1787826356.214156:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346319360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826356.214233:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346319360:12345-0@lo:400:kworker.0 Request processed in 83us (7024us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826356.214240:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8876 00000100:00100000:0.0:1787826356.214262:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8877 00000100:00100000:0.0:1787826356.214269:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346318976:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.214375:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346319744:12345-0@lo:400:kworker.0 Request processed in 1662us (7591us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826356.214384:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8875 00000100:00100000:3.0:1787826356.214400:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8878 00000100:00100000:3.0:1787826356.214409:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346319616:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1787826356.215440:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826356.215447:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826356.215450:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826356.215452:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826356.215487:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346318336 00000100:00100000:2.0:1787826356.215495:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346318336 00000100:00100000:2.0:1787826356.215538:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2239 00000100:00100000:2.0:1787826356.215546:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79515f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346318336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.215602:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79515f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346318336:12345-0@lo:400:kworker.0 Request processed in 57us (8754us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826356.215609:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2239 00000100:00100000:2.0:1787826356.215774:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346318336:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.215793:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346318848:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.215807:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346319360:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.215818:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346319872:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.215829:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346318720:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.215841:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346319232:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.215853:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346319744:0@lo:400:kworker.0 00000100:00100000:1.0:1787826356.216248:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346319488:12345-0@lo:400:kworker.0 Request processed in 4221us (8795us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826356.216258:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24062 00000100:00100000:2.0:1787826356.219482:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346318592:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.219513:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346319104:0@lo:400:kworker.0 00000100:00100000:3.0:1787826356.219580:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346319616:12345-0@lo:400:kworker.0 Request processed in 5171us (8328us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826356.219593:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8878 00000100:00100000:0.0:1787826356.219721:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346318976:12345-0@lo:400:kworker.0 Request processed in 5450us (12305us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826356.219729:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8877 00000100:00100000:2.0:1787826356.219937:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346319616:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.219954:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346318976:0@lo:400:kworker.0 00000100:00100000:2.0:1787826356.219966:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346319488:0@lo:400:kworker.0 00010000:00010000:0.0:1787826356.399587:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826356.446622:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826356.446751:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826356.447287:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332672128 00000100:00100000:2.0:1787826356.447298:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332672128 00000100:00080000:1.0:1787826356.447385:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00100000:2.0:1787826356.447385:0:8448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x523:0x0] (0->5) req@ffff9f785035bb80 x1874663332672128/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:712/0 lens 496/0 e 0 to 0 dl 1787826367 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826356.447398:0:8448:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785035bb80 with x1874663332672128 ext(0->5) 00002000:00010000:2.0:1787826356.447405:0:8448:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f785035bb80 00000100:00100000:2.0:1787826356.447439:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27664 00000100:00100000:2.0:1787826356.447449:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332672128:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:0.0:1787826356.449627:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332672256 00000100:00100000:0.0:1787826356.449643:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332672256 00000100:00100000:1.0:1787826356.450020:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30878 00000100:00100000:1.0:1787826356.450032:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2395:x1874663332672256:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:2.0:1787826356.450963:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2395:x1874663332672128:12345-192.168.203.56@tcp:4:bash.0 Request processed in 3508us (4205us total) trans 4294976170 rc 0/0 00000100:00100000:2.0:1787826356.450974:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27664 00002000:00100000:2.0:1787826356.450986:0:8448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x523:0x0] (0->5) req@ffff9f785035bb80 x1874663332672128/t4294976170(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:712/0 lens 496/448 e 0 to 0 dl 1787826367 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826356.450997:0:8448:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f785035bb80 with x1874663332672128 ext(0->5) 00002000:00010000:2.0:1787826356.451004:0:8448:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f785035bb80 00000001:00080000:1.0:1787826356.453614:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976170 is committed 00000001:00080000:1.0:1787826356.453633:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976169 is committed 00000100:00100000:2.0:1787826356.453935:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332672256:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 3899us (7319us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826356.453947:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30878 00000100:00100000:0.0:1787826356.478956:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826356.479012:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826356.479135:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332672512 00000100:00100000:2.0:1787826356.479151:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332672512 00000100:00080000:3.0:1787826356.479270:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826356.479321:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30879 00000100:00100000:2.0:1787826356.479345:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332672512:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:2.0:1787826356.479465:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332672512:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 133us (451us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826356.479473:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30879 00000100:00100000:1.0:1787826356.479781:0:8450:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332672384 00000100:00100000:1.0:1787826356.479787:0:8450:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332672384 00002000:00100000:1.0:1787826356.479819:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4d0:0x0] (0->5) req@ffff9f784b305180 x1874663332672384/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:712/0 lens 496/0 e 0 to 0 dl 1787826367 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826356.479829:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b305180 with x1874663332672384 ext(0->5) 00002000:00010000:1.0:1787826356.479835:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784b305180 00000100:00100000:1.0:1787826356.479863:0:8450:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27665 00000100:00100000:1.0:1787826356.479871:0:8450:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332672384:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:1.0:1787826356.481133:0:8450:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332672384:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1258us (2182us total) trans 4294973861 rc 0/0 00000100:00100000:1.0:1787826356.481144:0:8450:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27665 00002000:00100000:1.0:1787826356.481153:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4d0:0x0] (0->5) req@ffff9f784b305180 x1874663332672384/t4294973861(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:712/0 lens 496/448 e 0 to 0 dl 1787826367 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826356.481163:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784b305180 with x1874663332672384 ext(0->5) 00002000:00010000:1.0:1787826356.481167:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784b305180 00000100:00100000:3.0:1787826356.716416:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346320000:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826356.716497:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826356.716957:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346320000 00000100:00100000:0.0:1787826356.716965:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346320000 00000100:00100000:0.0:1787826356.717022:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6259 00000100:00100000:0.0:1787826356.717033:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346320000:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826356.717055:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683135488 00000100:00100000:0.0:1787826356.717197:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346320000:12345-0@lo:13:osp-pre-1-1.0 Request processed in 164us (699us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826356.717205:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6259 00000100:00100000:3.0:1787826356.717293:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976169 gen 2 00000004:00080000:3.0:1787826356.717299:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7842c91f80, transno 4294976169 00000100:00100000:3.0:1787826356.717326:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346320000:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826357.228867:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346320128:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826357.228948:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826357.229009:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346320256:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826357.229042:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826357.229062:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826357.229068:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346320384:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826357.229090:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826357.229143:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346320128 00000100:00100000:0.0:1787826357.229151:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346320128 00000100:00100000:0.0:1787826357.229502:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6260 00000100:00100000:0.0:1787826357.229511:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346320128:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787826357.229531:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826357.229567:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346320128:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (619us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826357.229574:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6260 00000100:00100000:0.0:1787826357.229606:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346320256 00000100:00100000:0.0:1787826357.229611:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346320256 00000100:00100000:0.0:1787826357.229633:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346320384 00000100:00100000:0.0:1787826357.229635:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346320384 00000100:00100000:0.0:1787826357.229653:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24063 00000100:00100000:0.0:1787826357.229658:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346320256:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1787826357.229682:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443364864 avail 1297125376 00000100:00100000:0.0:1787826357.229712:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346320256:12345-0@lo:41:osp-pre-1-0.0 Request processed in 54us (670us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826357.229718:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24063 00000100:00100000:0.0:1787826357.229730:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24064 00000100:00100000:0.0:1787826357.229734:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346320384:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787826357.229749:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442750464 avail 1296510976 00000100:00100000:0.0:1787826357.229769:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346320384:12345-0@lo:41:osp-pre-0-1.0 Request processed in 34us (678us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826357.229777:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24064 00000100:00100000:0.0:1787826357.229800:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850358380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346320512:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826357.229825:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826357.229850:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346320512 00000100:00100000:0.0:1787826357.229853:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346320512 00000100:00100000:0.0:1787826357.229871:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6261 00000100:00100000:0.0:1787826357.229877:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346320512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787826357.229887:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683135488 00000100:00100000:0.0:1787826357.229913:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346320512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 36us (88us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826357.229918:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6261 00000100:00100000:0.0:1787826357.229952:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850358380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346320512:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826357.229978:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346320384:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826357.229987:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346320256:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826357.230007:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346320128:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826357.232917:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826357.232948:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346320640:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826357.233025:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826357.233075:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346320640 00000100:00100000:0.0:1787826357.233083:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346320640 00000100:00100000:0.0:1787826357.233147:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6262 00000100:00100000:0.0:1787826357.233156:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346320640:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787826357.233179:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826357.233222:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346320640:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (197us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826357.233229:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6262 00000100:00100000:0.0:1787826357.233281:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346320640:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787826357.422545:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826358.446136:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826358.659661:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826358.659749:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332672640 00000100:00100000:1.0:1787826358.659766:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332672640 00000100:00100000:1.0:1787826358.659843:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68643 00000100:00100000:1.0:1787826358.659856:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226435:x1874663332672640:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826358.659880:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826358.659903:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c0ab lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a68 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826358.659973:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826358.659987:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826358.660000:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826358.660049:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826358.660053:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826358.660076:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826358.660102:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c0ab lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 8 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826358.660174:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826358.660182:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826358.660186:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826358.660189:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfa00, rc 0) 00000100:00100000:1.0:1787826358.660255:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226435:x1874663332672640:12345-192.168.203.56@tcp:101:cat.0 Request processed in 403us (603us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826358.660262:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68643 00000100:00080000:2.0:1787826358.663460:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826358.665101:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826358.666136:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332672768 00000100:00100000:2.0:1787826358.666150:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332672768 00000100:00100000:2.0:1787826358.666217:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68644 00000100:00100000:2.0:1787826358.666228:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226435:x1874663332672768:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826358.666248:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826358.666268:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c0b9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a6f expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666341:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666355:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666365:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666380:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666388:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666394:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666464:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c0ce lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666471:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c0ce lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666476:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c0ce lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826358.666497:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c0ce lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666512:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666519:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666525:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666531:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666536:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826358.666541:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666548:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666558:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826358.666561:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826358.666565:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666572:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c0c0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666585:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666590:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666596:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666601:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826358.666603:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666609:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666616:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7952ca9b00) 00010000:00010000:2.0:1787826358.666618:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826358.666620:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666630:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0c7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666660:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c0b9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a6f expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666673:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c0ce lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a6f expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.666680:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:2.0:1787826358.666683:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826358.666685:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826358.666755:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226435:x1874663332672768:12345-192.168.203.56@tcp:101:cat.0 Request processed in 529us (1666us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826358.666773:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68644 00000100:00100000:3.0:1787826358.671984:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826358.672033:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332672896 00000100:00100000:2.0:1787826358.672048:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332672896 00000100:00080000:3.0:1787826358.672139:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826358.672167:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68645 00000100:00100000:2.0:1787826358.672180:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226435:x1874663332672896:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826358.672200:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826358.672221:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c0d5 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a76 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.672286:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0dc lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.672298:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0dc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.672310:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0dc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826358.672421:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5a:0x0]: returning size 4096 00000004:00200000:2.0:1787826358.672427:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826358.672465:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c0d5 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a76 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.672482:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0dc lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a76 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826358.672491:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:2.0:1787826358.672494:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826358.672497:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826358.672565:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226435:x1874663332672896:12345-192.168.203.56@tcp:101:cat.0 Request processed in 387us (595us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826358.672572:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68645 00000100:00100000:2.0:1787826358.676431:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826358.676859:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332673024 00000100:00100000:0.0:1787826358.676869:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332673024 00000100:00100000:0.0:1787826358.676932:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68646 00000100:00100000:0.0:1787826358.676944:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226435:x1874663332673024:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826358.676965:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826358.676987:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c0e3 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a7d expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.677047:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.677061:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.677073:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826358.677151:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19b:0x0]: returning size 4096 00000004:00200000:0.0:1787826358.677158:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826358.677193:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c0e3 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.677210:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.677223:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:0.0:1787826358.677226:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826358.677229:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826358.677291:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226435:x1874663332673024:12345-192.168.203.56@tcp:101:cat.0 Request processed in 349us (869us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826358.677297:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68646 00000100:00100000:3.0:1787826358.682008:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826358.682229:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332673152 00000100:00100000:0.0:1787826358.682243:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332673152 00000100:00100000:0.0:1787826358.682312:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68647 00000100:00100000:0.0:1787826358.682337:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226435:x1874663332673152:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826358.682357:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826358.682383:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c0f1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a84 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.682451:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.682463:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.682474:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826358.682522:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x199:0x0]: returning size 4096 00000004:00200000:0.0:1787826358.682527:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826358.682568:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c0f1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.682590:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826358.682598:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826358.682601:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826358.682604:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826358.682714:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226435:x1874663332673152:12345-192.168.203.56@tcp:101:cat.0 Request processed in 392us (719us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826358.682730:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68647 00000100:00100000:1.0:1787826358.695826:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826358.695962:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332673408 00000100:00100000:3.0:1787826358.695980:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332673408 00000100:00100000:3.0:1787826358.696071:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68648 00000100:00100000:3.0:1787826358.696084:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226435:x1874663332673408:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826358.696146:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826358.696338:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c0ff lrc: 2/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a8b expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826358.696439:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826358.696456:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826358.696470:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826358.696484:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826358.696497:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826358.696505:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:3.0:1787826358.696599:0:24703:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000100:00100000:3.0:1787826359.188083:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826359.190488:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332673536 00000100:00100000:2.0:1787826359.190505:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332673536 00000100:00100000:2.0:1787826359.190617:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68649 00000100:00100000:2.0:1787826359.190627:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226437:x1874663332673536:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826359.190648:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826359.190671:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c114 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a92 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.190736:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.190750:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.190776:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826359.190826:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826359.190830:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826359.190853:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.190876:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c114 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.190896:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.190905:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826359.190908:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826359.190911:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef800, rc 0) 00000100:00100000:2.0:1787826359.190989:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226437:x1874663332673536:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 363us (2920us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.190996:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68649 00000100:00080000:3.0:1787826359.193039:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826359.194722:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826359.194953:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332673664 00000100:00100000:3.0:1787826359.194966:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332673664 00000100:00100000:3.0:1787826359.195032:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68650 00000100:00100000:3.0:1787826359.195045:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226437:x1874663332673664:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826359.195070:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826359.195095:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c122 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801a99 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195225:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195243:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195256:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195273:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195280:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195287:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195383:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195392:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195399:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826359.195422:0:8752:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195437:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195682:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195691:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195699:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195706:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826359.195720:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195730:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195745:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826359.195748:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826359.195753:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195761:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c129 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195779:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195785:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195791:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195796:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826359.195799:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195805:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195812:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79785cce00) 00010000:00010000:3.0:1787826359.195814:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826359.195817:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.195823:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c130 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.196428:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c122 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.196447:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.196455:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:3.0:1787826359.196457:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826359.196460:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826359.197183:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226437:x1874663332673664:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2134us (2478us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826359.197194:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68650 00000100:00100000:1.0:1787826359.210287:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826359.210589:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332673792 00000100:00100000:2.0:1787826359.210603:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332673792 00000100:00100000:2.0:1787826359.210748:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68651 00000100:00100000:2.0:1787826359.210762:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226437:x1874663332673792:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826359.211001:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826359.211100:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c13e lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801aa0 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.211273:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c145 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.211291:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c145 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.211307:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c145 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826359.211434:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5a:0x0]: returning size 4096 00000004:00200000:2.0:1787826359.211440:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826359.211487:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c13e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aa0 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.211511:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c145 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aa0 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.211522:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:2.0:1787826359.211525:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826359.211530:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826359.211655:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226437:x1874663332673792:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 894us (1381us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.211663:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68651 00000100:00080000:3.0:1787826359.211747:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826359.217626:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826359.217788:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332673920 00000100:00100000:2.0:1787826359.217806:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332673920 00000100:00100000:2.0:1787826359.217886:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68652 00000100:00100000:2.0:1787826359.217900:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226437:x1874663332673920:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826359.217928:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826359.217953:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c14c lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801aa7 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218058:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218076:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218091:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218135:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218146:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218154:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218491:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4c161 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218502:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4c161 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218508:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4c161 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826359.218556:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5b:0x0]: returning size 6 00000004:00200000:2.0:1787826359.218562:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826359.218596:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4c161 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218609:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4c161 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218619:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218626:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218633:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218639:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218645:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826359.218649:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218657:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218671:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826359.218673:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826359.218678:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218684:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c153 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218704:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218710:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218715:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218720:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826359.218723:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218728:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218734:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x4ec94fd6 (ffff9f79785cc900) 00010000:00010000:2.0:1787826359.218736:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826359.218739:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218745:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee200/0xe21309aa81f4c15a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218774:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c14c lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aa7 expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218787:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4c161 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aa7 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.218800:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:2.0:1787826359.218802:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826359.218804:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826359.219328:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226437:x1874663332673920:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1418us (1704us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.219337:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68652 00000100:00100000:1.0:1787826359.236250:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826359.236737:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332674048 00000100:00100000:0.0:1787826359.236751:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332674048 00000100:00100000:0.0:1787826359.236853:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68653 00000100:00100000:0.0:1787826359.236864:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226437:x1874663332674048:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826359.236891:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826359.236915:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c168 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801aae expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.236994:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237021:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237036:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237054:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237061:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237067:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237209:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c17d lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237220:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c17d lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237228:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c17d lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826359.237276:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dc1:0x0]: returning size 6 00000004:00200000:0.0:1787826359.237282:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826359.237304:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c17d lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237327:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c17d lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237339:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237347:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237359:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237366:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237373:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826359.237379:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237386:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237407:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:0.0:1787826359.237410:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826359.237416:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 3/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237423:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c16f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237440:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237447:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237453:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237458:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826359.237461:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237467:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237475:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785cc600) 00010000:00010000:0.0:1787826359.237478:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826359.237481:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237487:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c176 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237516:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c168 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aae expref: 13 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237532:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c17d lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aae expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.237540:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:0.0:1787826359.237542:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826359.237545:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:0.0:1787826359.239100:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226437:x1874663332674048:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2224us (2853us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.239143:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68653 00000100:00100000:1.0:1787826359.245690:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826359.245760:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826359.246339:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332674176 00000100:00100000:3.0:1787826359.246352:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332674176 00000100:00100000:3.0:1787826359.246437:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11198 00000100:00100000:3.0:1787826359.246451:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:2395:x1874663332674176:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:3.0:1787826359.246471:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826359.246501:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eec00/0xe21309aa81f4c161 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aa7 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.246526:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f7952ca9b00) 00010000:00010000:3.0:1787826359.246539:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826359.246623:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:2395:x1874663332674176:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 174us (952us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826359.246632:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11198 00000100:00100000:0.0:1787826359.247307:0:6542:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332674432 00000100:00100000:0.0:1787826359.247326:0:6542:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332674432 00000100:00100000:0.0:1787826359.247385:0:6542:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1236 00000100:00100000:0.0:1787826359.247394:0:6542:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226437:x1874663332674432:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:0.0:1787826359.247452:0:6542:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826359.247473:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4c17d lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aae expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.247489:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f784b4a5200) 00010000:00010000:0.0:1787826359.247498:0:6542:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826359.247599:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346320768:0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826359.247673:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826359.247747:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.247770:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346320768 00000100:00100000:0.0:1787826359.248280:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346320768 00000100:00100000:0.0:1787826359.248337:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24065 00000100:00100000:0.0:1787826359.248347:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346320768:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826359.252575:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346320768:12345-0@lo:1000:mdt_io00_002.0 Request processed in 4215us (4886us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.252594:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24065 00000100:00100000:3.0:1787826359.253134:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346320768:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826359.253260:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346320896:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826359.253342:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826359.253386:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.253608:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346320896 00000100:00100000:0.0:1787826359.254370:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346320896 00000100:00100000:0.0:1787826359.254439:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24066 00000100:00100000:0.0:1787826359.254451:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346320896:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826359.254586:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346320896:12345-0@lo:1000:mdt_io00_002.0 Request processed in 136us (1243us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.254595:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24066 00000100:00100000:1.0:1787826359.254856:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b304000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346320896:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826359.254901:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19b:0x0] 00000100:00100000:1.0:1787826359.254950:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346321024:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.254990:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.255041:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826359.255338:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346321024 00000100:00100000:3.0:1787826359.255344:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346321024 00000100:00100000:3.0:1787826359.255387:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24067 00000100:00100000:3.0:1787826359.255396:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346321024:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826359.255468:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346321024:12345-0@lo:1000:mdt_io00_002.0 Request processed in 74us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826359.255474:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24067 00000100:00100000:1.0:1787826359.255564:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346321024:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826359.255976:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5a:0x0] 00000100:00100000:1.0:1787826359.256247:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346321152:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.256284:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.256326:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.256396:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346321152 00000100:00100000:0.0:1787826359.256402:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346321152 00000100:00100000:0.0:1787826359.256443:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24068 00000100:00100000:0.0:1787826359.256452:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346321152:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826359.256522:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346321152:12345-0@lo:1000:mdt_io00_002.0 Request processed in 71us (237us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.256529:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24068 00000100:00100000:1.0:1787826359.257416:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346321152:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826359.257477:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19b:0x0] 00000100:00100000:1.0:1787826359.257514:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346321280:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.257553:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.257607:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.258992:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346321280 00000100:00100000:0.0:1787826359.259006:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346321280 00000100:00100000:0.0:1787826359.259535:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24069 00000100:00100000:0.0:1787826359.259548:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346321280:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826359.259703:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346321280:12345-0@lo:1000:mdt_io00_002.0 Request processed in 156us (2148us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.259711:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24069 00000100:00100000:1.0:1787826359.259980:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346321280:0@lo:1000:mdt_io00_002.0 00000004:00020000:1.0:1787826359.260457:0:6542:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:1.0:1787826359.260518:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.260531:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.260542:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826359.260565:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346321408:0@lo:101:mdt_io00_002.0 00000100:00100000:1.0:1787826359.260632:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.260673:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000004:00020000:2.0:1787826359.261151:0:24703:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4434 00010000:00010000:2.0:1787826359.261225:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261243:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261257:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826359.261272:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dc1:0x0]: returning size 6 00010000:00010000:2.0:1787826359.261391:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261401:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261408:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261416:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261423:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826359.261429:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261438:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261454:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826359.261458:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826359.261463:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261471:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c106 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261489:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261496:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261503:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261509:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826359.261512:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261520:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261528:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785cc600) 00010000:00010000:2.0:1787826359.261531:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826359.261535:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261542:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c10d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261606:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c0ff lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 12 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261631:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 11 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.261640:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:2.0:1787826359.261643:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826359.261646:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826359.261712:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226435:x1874663332673408:12345-192.168.203.56@tcp:101:cat.0 Request processed in 565631us (565896us total) trans 8589957473 rc 0/0 00000100:00100000:2.0:1787826359.261731:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68648 00000100:00100000:0.0:1787826359.262237:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346321408 00000100:00100000:0.0:1787826359.262247:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346321408 00000100:00100000:0.0:1787826359.262298:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24070 00000100:00100000:0.0:1787826359.262309:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346321408:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826359.262337:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826359.262355:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c184 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.262377:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 2/0,1 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.262388:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.262397:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 5/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.262430:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.262446:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 10 pid: 35095 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.266493:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346321536:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826359.266623:0:6532:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:00080000:0.0:1787826359.266668:0:6532:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00000100:00100000:0.0:1787826359.266722:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.268057:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826359.271585:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332674560 00000100:00100000:2.0:1787826359.271603:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332674560 00000100:00100000:2.0:1787826359.271686:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30880 00000100:00100000:2.0:1787826359.271699:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332674560:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826359.271723:0:8759:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826359.271748:0:8759:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4c1a0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801ab5 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826359.271819:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8759:1874663346321664:192.168.203.56@tcp:106: 00000100:00100000:2.0:1787826359.271932:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703d80 going to sleep for 11 seconds 00000100:00080000:2.0:1787826359.271952:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826359.274402:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6532:1874663346321536:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826359.274454:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.274469:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:1.0:1787826359.274473:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826359.274477:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 7 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.274489:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:1.0:1787826359.274581:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346321408:12345-0@lo:101:mdt_io00_002.0 Request processed in 12272us (13948us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.274590:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24070 00000100:00100000:0.0:1787826359.274727:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f686a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346321408:0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826359.274750:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.274763:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.275530:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826359.277795:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332674688 00000100:00100000:3.0:1787826359.277813:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332674688 00010000:00010000:3.0:1787826359.277878:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 11 pid: 35095 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826359.277922:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22049 00000100:00100000:3.0:1787826359.277935:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332674688:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826359.277953:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826359.277961:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 11 pid: 35095 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.277976:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 11 pid: 35095 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.278000:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c0ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a7d expref: 11 pid: 35095 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.278023:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:3.0:1787826359.278032:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 6 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.278045:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826359.278149:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332674688:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 215us (2622us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826359.278157:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22049 00002000:00010000:3.0:1787826359.283244:0:8759:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x2c0000401:0x523:0x0] updating lvb size: 0 -> 6 00002000:00010000:3.0:1787826359.283255:0:8759:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x2c0000401:0x523:0x0] updating lvb mtime: 0 -> 1787826353 00002000:00010000:3.0:1787826359.283259:0:8759:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x2c0000401:0x523:0x0] updating lvb atime: 0 -> 1787826353 00002000:00010000:3.0:1787826359.283261:0:8759:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x2c0000401:0x523:0x0] updating lvb ctime: 0 -> 1787826353 00002000:00010000:3.0:1787826359.283263:0:8759:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x2c0000401:0x523:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:3.0:1787826359.283292:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8759:1874663346321664:192.168.203.56@tcp:106: 00010000:00010000:3.0:1787826359.283354:0:8759:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4c1a0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ab5 expref: 6 pid: 8759 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.283383:0:8759:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e38b753e/0xe21309aa81f4c1a0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ab5 expref: 6 pid: 8759 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.283398:0:8759:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee200, rc 0) 00000100:00100000:3.0:1787826359.283474:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332674560:12345-192.168.203.56@tcp:101:cat.0 Request processed in 11777us (15425us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826359.283484:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30880 00000001:00080000:3.0:1787826359.286669:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985472 is committed 00000020:00080000:3.0:1787826359.286697:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7850705900 refcount 2 committed 0 result 0 batchid 3441 00000020:00080000:3.0:1787826359.286704:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796efc4900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7850705d80 00000020:00080000:3.0:1787826359.286708:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1545 00000020:00080000:3.0:1787826359.286711:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f797cc62c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a236200 00000020:00080000:3.0:1787826359.286714:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1227 00000020:00080000:3.0:1787826359.286732:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3438 batchid updated 00000001:00080000:3.0:1787826359.286740:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985469 is committed 00000020:00080000:3.0:1787826359.286744:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784631a840 refcount 2 committed 0 result 0 batchid 3440 00000020:00080000:3.0:1787826359.286747:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78433eac00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784631a180 00000020:00080000:3.0:1787826359.286750:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1544 00000020:00080000:3.0:1787826359.286753:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796d9b9300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784dac5000 00000020:00080000:3.0:1787826359.286756:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1226 00000001:00080000:3.0:1787826359.286764:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985467 is committed 00000020:00080000:3.0:1787826359.286774:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784afb7000 refcount 2 committed 0 result 0 batchid 3439 00000020:00080000:3.0:1787826359.286777:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784b0ac500 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784afb7a80 00000020:00080000:3.0:1787826359.286780:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1543 00000020:00080000:3.0:1787826359.286783:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784b0ac100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852be1800 00000020:00080000:3.0:1787826359.286786:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1225 00000020:00080000:1.0:1787826359.287069:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3441 committed batchid 3438 00000004:00080000:1.0:1787826359.287236:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7850705d80 with version 6884 00000020:00080000:1.0:1787826359.287245:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3438 cancel update log [0x200000401:0x3:0x0].1542: rc = 0 00000020:00080000:1.0:1787826359.287298:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3438 cancel update log [0x240000400:0x3:0x0].1224: rc = 0 00000020:00080000:1.0:1787826359.287327:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3438 00010000:00010000:0.0:1787826359.287857:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287873:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287882:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287890:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826359.287897:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287906:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287927:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:0.0:1787826359.287931:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826359.287936:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287944:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee400/0xe21309aa81f4bfe7 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287963:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287970:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287976:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287982:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826359.287985:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287991:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.287998:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:0.0:1787826359.288000:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826359.288008:0:3638:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 5 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.288021:0:3638:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 5 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.288032:0:3638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 5 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.288091:0:3638:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 5 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.288133:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848300380 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0001_UUID:3638:1874663346321792:0@lo:105: 00000100:00100000:0.0:1787826359.288213:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826359.288269:0:3638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4d80 going to sleep for 11 seconds 00000004:00080000:2.0:1787826359.288920:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6884 rpc_version 6884 00000020:00100000:2.0:1787826359.288955:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826359.288965:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784ade3b80 x1874663346322048/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826359.288986:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346322048:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826359.289050:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826359.289083:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.290161:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346321792 00000100:00100000:1.0:1787826359.290177:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346321792 00000100:00100000:1.0:1787826359.290253:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7917 00000100:00100000:1.0:1787826359.290278:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:3638:x1874663346321792:12345-0@lo:105: 00010000:00010000:1.0:1787826359.290297:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290311:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290332:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290344:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290353:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290364:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290406:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ece800) 00000100:00100000:1.0:1787826359.290487:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:3638:x1874663346321792:12345-0@lo:105: Request processed in 211us (2273us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.290495:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7917 00000100:00100000:1.0:1787826359.290546:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848300380 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0001_UUID:3638:1874663346321792:0@lo:105: 00010000:00010000:1.0:1787826359.290576:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290598:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4bfee lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290622:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290630:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290638:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290644:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826359.290649:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290658:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290674:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x4ec94fd6 (ffff9f79785cc900) 00010000:00010000:1.0:1787826359.290678:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826359.290682:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290688:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c08f lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290719:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290729:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826359.290755:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19b:0x0] 00010000:00010000:1.0:1787826359.290802:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290810:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.290821:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826359.290852:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322176:0@lo:101:mdt_io00_002.0 00000100:00100000:1.0:1787826359.290892:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.290915:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.291000:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346322048 00000100:00100000:0.0:1787826359.291007:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346322048 00000100:00100000:0.0:1787826359.291250:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346322176 00000100:00100000:0.0:1787826359.291255:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346322176 00000100:00100000:0.0:1787826359.291290:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24071 00000100:00100000:0.0:1787826359.291299:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346322048:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826359.291509:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346322048:12345-0@lo:1000:osp_up0-1.0 Request processed in 210us (2457us total) trans 8589957474 rc 0/0 00000100:00100000:0.0:1787826359.291518:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24071 00000100:00100000:0.0:1787826359.291536:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24072 00000100:00100000:0.0:1787826359.291543:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6542:x1874663346322176:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787826359.291553:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826359.291571:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c1ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c1a7 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.291608:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c1ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c1a7 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.291622:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c1ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c1a7 expref: 7 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.291630:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x4ec94fd6 (ffff9f784b4a5200) 00010000:00010000:0.0:1787826359.291633:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826359.291637:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826359.291664:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322176:12345-0@lo:101:mdt_io00_002.0 Request processed in 122us (772us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.291670:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24072 00000100:00100000:0.0:1787826359.291726:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346322048:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826359.292600:0:6542:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985467 gen 2 00000004:00080000:1.0:1787826359.292609:0:6542:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985467 committed transno 4294985467 00000020:00080000:1.0:1787826359.292617:0:6542:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f78434e4c00 refcount 2 committed 0 result 0 batchid 4294979088 00000020:00080000:1.0:1787826359.292621:0:6542:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f79460ac800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78434e4240 00000020:00080000:1.0:1787826359.292625:0:6542:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].945 00000020:00080000:1.0:1787826359.292628:0:6542:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784b0ac200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7852be0400 00000020:00080000:1.0:1787826359.292631:0:6542:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].942 00000100:00100000:1.0:1787826359.292673:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322176:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787826359.292694:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c1ae expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.292708:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c1ae expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826359.292728:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19b:0x0] 00010000:00010000:1.0:1787826359.292751:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.292767:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.292776:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.292811:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.292827:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 10 pid: 35080 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826359.292858:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346322304:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826359.292947:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.294092:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000020:00080000:0.0:1787826359.294400:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979088 committed batchid 4294979087 00000020:00080000:0.0:1787826359.294479:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979087 00000100:00100000:3.0:1787826359.297741:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332674816 00000100:00100000:3.0:1787826359.297763:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332674816 00000100:00100000:0.0:1787826359.299457:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30881 00000100:00100000:0.0:1787826359.299472:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:226435:x1874663332674816:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826359.299497:0:24218:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826359.299519:0:24218:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801abc expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.299557:0:24218:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.299566:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x523:0x0].0x0 since=5s v=(0 0) 00010000:00010000:0.0:1787826359.299571:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826359.299577:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.299626:0:24218:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 5 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.299639:0:24218:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 5 pid: 8443 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.299654:0:24218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848301f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:24218:1874663346322432:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826359.299822:0:24218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826359.303655:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826359.303713:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332674944 00000100:00100000:2.0:1787826359.303724:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332674944 00010000:00010000:2.0:1787826359.303776:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 11 pid: 35080 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826359.303818:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22050 00000100:00100000:2.0:1787826359.303984:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332674944:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826359.304006:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826359.304023:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 11 pid: 35080 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.304039:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 11 pid: 35080 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.304066:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:2.0:1787826359.304074:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.304085:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.304093:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.304189:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826359.304355:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332674944:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 524us (712us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.304363:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22050 00010000:00010000:3.0:1787826359.306428:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c0f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a84 expref: 10 pid: 35080 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826359.306456:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641180 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346322304:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826359.306486:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.306497:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.306503:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.306529:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.306542:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.306552:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826359.306743:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322560:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826359.306814:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826359.306923:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.307141:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346322560 00000100:00100000:1.0:1787826359.307151:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346322560 00000100:00100000:1.0:1787826359.307211:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24073 00000100:00100000:1.0:1787826359.307224:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322560:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.307458:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322560:12345-0@lo:1000:mdt_io00_002.0 Request processed in 234us (642us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.307467:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24073 00000100:00100000:1.0:1787826359.307507:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322560:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.307569:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322688:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.309704:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.309749:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.310174:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346322688 00000100:00100000:0.0:1787826359.310186:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346322688 00000100:00100000:0.0:1787826359.310239:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24074 00000100:00100000:0.0:1787826359.310251:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322688:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.310833:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826359.311365:0:24218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848301f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:24218:1874663346322432:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826359.311674:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332675072 00000100:00100000:2.0:1787826359.311682:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332675072 00010000:00010000:2.0:1787826359.311728:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 6 pid: 8443 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826359.311765:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22051 00000100:00100000:2.0:1787826359.311776:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c80450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332675072:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826359.311790:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826359.311814:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 6 pid: 8443 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.311830:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 6 pid: 8443 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.312239:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322688:12345-0@lo:1000:mdt_io00_002.0 Request processed in 1988us (2542us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.312249:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24074 00000100:00080000:1.0:1787826359.312356:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826359.312402:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322688:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.312486:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322816:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.312531:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.312569:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.312973:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346322816 00000100:00100000:0.0:1787826359.312981:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346322816 00000100:00100000:0.0:1787826359.313036:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24075 00000100:00100000:0.0:1787826359.313046:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322816:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826359.313211:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322816:12345-0@lo:1000:mdt_io00_002.0 Request processed in 167us (679us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826359.313219:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24075 00000100:00100000:1.0:1787826359.313626:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322816:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.313718:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322944:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.313751:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.313786:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.313854:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346322944 00000100:00100000:0.0:1787826359.313860:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346322944 00000100:00100000:0.0:1787826359.313899:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24076 00000100:00100000:0.0:1787826359.313907:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322944:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826359.313974:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346322944:12345-0@lo:1000:mdt_io00_002.0 Request processed in 67us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.313981:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24076 00000100:00100000:1.0:1787826359.315181:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346322944:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.315307:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323072:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.315363:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.315400:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.315512:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346323072 00000100:00100000:0.0:1787826359.315520:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346323072 00000100:00100000:0.0:1787826359.315567:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24077 00000100:00100000:0.0:1787826359.315577:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346323072:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787826359.315670:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346323072:12345-0@lo:1000:mdt_io00_002.0 Request processed in 94us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.315679:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24077 00000100:00100000:1.0:1787826359.316986:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323072:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826359.317043:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5a:0x0] 00010000:00010000:1.0:1787826359.317207:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.317229:0:6542:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.317239:0:6542:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.317676:0:6542:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.317700:0:6542:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 9 pid: 24703 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826359.317716:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346323200:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826359.317890:0:24218:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.317916:0:24218:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:1.0:1787826359.320571:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705a80 going to sleep for 11 seconds 00010000:00010000:2.0:1787826359.321351:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c07a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a53 expref: 6 pid: 8443 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.321378:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826359.321383:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826359.321395:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.321407:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.321476:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826359.321494:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6511:1874663346323328:192.168.203.56@tcp:105: 00000100:00100000:2.0:1787826359.321594:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850703cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826359.322409:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226435:x1874663332674816:12345-192.168.203.56@tcp:101:cat.0 Request processed in 22935us (28319us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.322422:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30881 00000100:00100000:0.0:1787826359.333824:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826359.333871:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826359.333915:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332675328 00000100:00100000:0.0:1787826359.333927:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332675328 00010000:00010000:0.0:1787826359.333975:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 10 pid: 24703 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.334144:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22052 00000100:00100000:0.0:1787826359.334157:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332675328:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826359.334176:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826359.334184:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 5/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 10 pid: 24703 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.334204:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 10 pid: 24703 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.334230:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:0.0:1787826359.334236:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.334246:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.334253:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.334336:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826359.334454:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332675328:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 299us (581us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.334464:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22052 00000100:00100000:3.0:1787826359.335741:0:24159:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332675200 00000100:00100000:3.0:1787826359.335756:0:24159:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332675200 00002000:00100000:3.0:1787826359.335818:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_010: refresh rw locks for [0x2c0000401:0x523:0x0] (0->4095) req@ffff9f7848301f80 x1874663332675200/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:715/0 lens 488/0 e 0 to 0 dl 1787826370 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826359.335831:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7848301f80 with x1874663332675200 ext(0->4095) 00002000:00010000:3.0:1787826359.335850:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7848301f80 00000100:00100000:3.0:1787826359.335885:0:24159:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27666 00000100:00100000:3.0:1787826359.335896:0:24159:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332675200:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:3.0:1787826359.336175:0:24159:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332675200:12345-192.168.203.56@tcp:3:cat.0 Request processed in 279us (2363us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787826359.336184:0:24159:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27666 00002000:00100000:3.0:1787826359.336192:0:24159:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_010: refresh rw locks for [0x2c0000401:0x523:0x0] (0->4095) req@ffff9f7848301f80 x1874663332675200/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:715/0 lens 488/4536 e 0 to 0 dl 1787826370 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826359.336201:0:24159:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7848301f80 with x1874663332675200 ext(0->4095) 00002000:00010000:3.0:1787826359.336205:0:24159:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f7848301f80 00000100:00100000:2.0:1787826359.336998:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6511:1874663346323328:192.168.203.56@tcp:105: 00010000:00010000:2.0:1787826359.337091:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826359.337157:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c80450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332675072:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 25381us (26327us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.337167:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22051 00010000:00010000:0.0:1787826359.337354:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4c18b lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a8b expref: 9 pid: 24703 timeout: 8271 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.337377:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6542:1874663346323200:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826359.337399:0:6542:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.337410:0:6542:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.337416:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.337484:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.337492:0:6542:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.337503:0:6542:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.337527:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323456:0@lo:101:mdt_io00_002.0 00000100:00100000:0.0:1787826359.337580:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826359.337736:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.341603:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826359.342444:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332675456 00000100:00100000:3.0:1787826359.342459:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332675456 00000100:00100000:3.0:1787826359.342541:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68654 00000100:00100000:3.0:1787826359.342554:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226435:x1874663332675456:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826359.342597:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826359.342622:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c1d8 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801ac3 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.342693:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c1d8 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ac3 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.342711:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c1d8 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ac3 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.342734:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:3.0:1787826359.342739:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787826359.342743:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826359.342825:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226435:x1874663332675456:12345-192.168.203.56@tcp:101:cat.0 Request processed in 273us (1242us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826359.348962:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68654 00000100:00100000:1.0:1787826359.349781:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826359.349957:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332675712 00000100:00100000:3.0:1787826359.349971:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332675712 00000100:00100000:3.0:1787826359.350034:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9389 00000100:00100000:3.0:1787826359.350047:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226435:x1874663332675712:12345-192.168.203.56@tcp:35:cat.0 00000100:00100000:3.0:1787826359.350441:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226435:x1874663332675712:12345-192.168.203.56@tcp:35:cat.0 Request processed in 394us (663us total) trans 8589957475 rc 0/0 00000100:00100000:3.0:1787826359.350450:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9389 00000100:00100000:2.0:1787826359.350600:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346323456 00000100:00100000:2.0:1787826359.350607:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346323456 00000100:00100000:2.0:1787826359.350641:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24078 00000100:00100000:2.0:1787826359.350649:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6542:x1874663346323456:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787826359.350661:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826359.350690:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c1df lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c1d1 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.350727:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c1df lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c1d1 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.350743:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c1df lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c1d1 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.350753:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f784abfe200) 00010000:00010000:2.0:1787826359.350756:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787826359.350760:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826359.350826:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323456:12345-0@lo:101:mdt_io00_002.0 Request processed in 178us (13245us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.350832:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24078 00000100:00100000:3.0:1787826359.351073:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323456:0@lo:101:mdt_io00_002.0 00010000:00010000:3.0:1787826359.351095:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c1df expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.351126:0:6542:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c1df expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826359.351156:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5b:0x0] 00000100:00100000:3.0:1787826359.351222:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323584:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826359.351268:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826359.351303:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.351374:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346323584 00000100:00100000:1.0:1787826359.351380:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346323584 00000100:00100000:1.0:1787826359.351423:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24079 00000100:00100000:1.0:1787826359.351432:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323584:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.351529:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323584:12345-0@lo:1000:mdt_io00_002.0 Request processed in 98us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.351536:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24079 00000100:00100000:3.0:1787826359.351856:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323584:0@lo:1000:mdt_io00_002.0 00000004:00080000:3.0:1787826359.351886:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19b:0x0] 00000100:00100000:3.0:1787826359.351914:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323712:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787826359.351942:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826359.351970:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.352024:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346323712 00000100:00100000:1.0:1787826359.352029:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346323712 00000100:00100000:1.0:1787826359.352064:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24080 00000100:00100000:1.0:1787826359.352071:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323712:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.352158:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323712:12345-0@lo:1000:mdt_io00_002.0 Request processed in 87us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.352166:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24080 00000100:00100000:1.0:1787826359.352191:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323712:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826359.352207:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5a:0x0] 00010000:00010000:1.0:1787826359.352244:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.352257:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.352266:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826359.352309:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323840:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.352351:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.352368:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.352380:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346323840 00000100:00100000:1.0:1787826359.352384:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346323840 00000100:00100000:1.0:1787826359.352407:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24081 00000100:00100000:1.0:1787826359.352414:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323840:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.352465:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323840:12345-0@lo:1000:mdt_io00_002.0 Request processed in 51us (113us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.352472:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24081 00000100:00100000:1.0:1787826359.352497:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323840:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826359.352519:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5a:0x0] 00000100:00100000:1.0:1787826359.352539:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323968:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.352564:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.352577:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.352660:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346323968 00000100:00100000:1.0:1787826359.352664:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346323968 00000100:00100000:1.0:1787826359.352682:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24082 00000100:00100000:1.0:1787826359.352688:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323968:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.352738:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346323968:12345-0@lo:1000:mdt_io00_002.0 Request processed in 50us (173us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.352745:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24082 00000100:00100000:1.0:1787826359.352768:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346323968:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.352801:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324096:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.352825:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.352838:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.352848:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346324096 00000100:00100000:1.0:1787826359.352852:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346324096 00000100:00100000:1.0:1787826359.352868:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24083 00000100:00100000:1.0:1787826359.352874:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324096:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.352919:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324096:12345-0@lo:1000:mdt_io00_002.0 Request processed in 44us (93us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.352926:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24083 00000100:00100000:1.0:1787826359.352948:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324096:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826359.352963:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19b:0x0] 00000100:00100000:1.0:1787826359.352983:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324224:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353010:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.353024:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.353034:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346324224 00000100:00100000:1.0:1787826359.353037:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346324224 00000100:00100000:1.0:1787826359.353055:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24084 00000100:00100000:1.0:1787826359.353060:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324224:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353132:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324224:12345-0@lo:1000:mdt_io00_002.0 Request processed in 71us (121us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.353139:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24084 00000100:00100000:1.0:1787826359.353164:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324224:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787826359.353182:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5a:0x0] 00000100:00100000:1.0:1787826359.353207:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324352:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353232:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.353245:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.353256:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346324352 00000100:00100000:1.0:1787826359.353260:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346324352 00000100:00100000:1.0:1787826359.353276:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24085 00000100:00100000:1.0:1787826359.353281:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324352:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353339:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324352:12345-0@lo:1000:mdt_io00_002.0 Request processed in 57us (106us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.353346:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24085 00000100:00100000:1.0:1787826359.353369:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324352:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353396:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324480:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353421:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.353435:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d00e480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.353447:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346324480 00000100:00100000:1.0:1787826359.353451:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346324480 00000100:00100000:1.0:1787826359.353468:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24086 00000100:00100000:1.0:1787826359.353474:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324480:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353519:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324480:12345-0@lo:1000:mdt_io00_002.0 Request processed in 44us (96us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826359.353525:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24086 00000100:00100000:1.0:1787826359.353548:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324480:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353937:0:6542:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324608:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787826359.353965:0:6542:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826359.354030:0:6542:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826359.357413:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346324608 00000100:00100000:2.0:1787826359.357435:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346324608 00000100:00100000:2.0:1787826359.357518:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24087 00000100:00100000:2.0:1787826359.357531:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324608:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787826359.357719:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6542:x1874663346324608:12345-0@lo:1000:mdt_io00_002.0 Request processed in 190us (3752us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.357727:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24087 00000100:00100000:1.0:1787826359.360331:0:6542:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837b480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6542:1874663346324608:0@lo:1000:mdt_io00_002.0 00000040:00080000:1.0:1787826359.361254:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].216, 64 off22016 00000004:00080000:1.0:1787826359.361266:0:6542:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8171573160019ns, next at 8172573157444ns, rc = 0 00000040:00080000:1.0:1787826359.361312:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].943, 904 off887264 00000040:00080000:1.0:1787826359.361797:0:6542:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].946, 984 off987128 00000004:00080000:1.0:1787826359.361805:0:6542:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7850705240 with version 1838 00000004:00080000:3.0:1787826359.361976:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1838 rpc_version 1838 00000020:00100000:3.0:1787826359.362022:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826359.362035:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78435b3100 x1874663346324864/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2584/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826359.362063:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b3100 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346324864:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826359.362183:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826359.362227:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826359.362296:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346324864 00000100:00100000:1.0:1787826359.362307:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346324864 00000100:00100000:1.0:1787826359.362388:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24088 00000100:00100000:1.0:1787826359.362400:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346324864:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826359.363057:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346324864:12345-0@lo:1000:osp_up1-0.0 Request processed in 657us (871us total) trans 4294985473 rc 0/0 00000100:00100000:1.0:1787826359.363066:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24088 00000100:00100000:3.0:1787826359.364367:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b3100 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346324864:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787826359.364859:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.364874:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.364883:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.364891:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826359.364897:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.364906:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.364925:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:1.0:1787826359.364929:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826359.364934:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.364940:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c1ca lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826359.364962:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5b:0x0] 00010000:00010000:1.0:1787826359.364969:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c1df expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.364984:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c1df expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.364996:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365002:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365008:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365014:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826359.365017:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365022:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365030:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f784b4a5400) 00010000:00010000:1.0:1787826359.365032:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826359.365035:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365041:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1b5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826359.365062:0:6542:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000066058f7 reply state 0000000015856caa transno 8589957476 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365081:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365090:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785cc600) 00010000:00010000:1.0:1787826359.365093:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826359.365096:0:6542:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:1.0:1787826359.365099:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19b:0x0] 00010000:00010000:1.0:1787826359.365146:0:6542:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365155:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365162:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365192:0:6542:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826359.365197:0:6542:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19b:0x0] 00010000:00010000:1.0:1787826359.365201:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c1ae expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365208:0:6542:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c1ae expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365216:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365222:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365228:0:6542:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365233:0:6542:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826359.365236:0:6542:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365241:0:6542:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365248:0:6542:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f7845cb1700) 00010000:00010000:1.0:1787826359.365251:0:6542:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826359.365254:0:6542:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365259:0:6542:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c1e6 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365289:0:6542:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365300:0:6542:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365307:0:6542:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365325:0:6542:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365332:0:6542:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365338:0:6542:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826359.365361:0:6542:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1787826359.365475:0:6542:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f687800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:226437:x1874663332674432:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 118076us (119710us total) trans 8589957476 rc 0/0 00000100:00100000:1.0:1787826359.365484:0:6542:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1236 00010000:00010000:3.0:1787826359.366138:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826359.366146:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366156:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826359.366161:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826359.366170:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366213:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366220:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826359.366247:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366260:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ece800/0xe21309aa81f4c184 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c199 expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366279:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826359.366284:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366290:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826359.366294:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366301:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366312:0:6514:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:3.0:1787826359.366326:0:6514:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787826359.366330:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366336:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c1ed lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dc1:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.366345:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826359.366347:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826359.366349:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826359.369025:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346324992:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826359.369334:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826359.369418:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826359.381794:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346324992 00000100:00100000:2.0:1787826359.381818:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346324992 00000100:00100000:2.0:1787826359.381926:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11199 00000100:00100000:2.0:1787826359.381938:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346324992:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826359.381979:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826359.382013:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c199 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c184 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.382043:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7967ac8200) 00010000:00010000:2.0:1787826359.382057:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826359.382191:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346324992:12345-0@lo:103:ldlm_bl.0 Request processed in 252us (12772us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826359.382199:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11199 00000100:00100000:0.0:1787826359.382807:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848302680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346324992:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826359.468427:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826359.791894:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c1ae expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826359.791920:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c1ae expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.791972:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c1ae expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.791980:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826359.792005:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c1a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c1ae expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792036:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c1df expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826359.792044:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c1df expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792062:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c1df expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792069:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826359.792081:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d42000/0xe21309aa81f4c1d1 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c1df expref: -99 pid: 6542 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826359.792096:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957476 is committed 00000004:00080000:0.0:1787826359.792123:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 221 committed 00000001:00080000:0.0:1787826359.792140:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957475 is committed 00000020:00080000:0.0:1787826359.792149:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979088 batchid updated 00000001:00080000:0.0:1787826359.792160:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957474 is committed 00010000:00010000:0.0:1787826359.792190:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792199:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792207:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792214:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826359.792220:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792228:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792244:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785cc600) 00010000:00010000:0.0:1787826359.792248:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826359.792253:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792260:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c1c3 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6542 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.792297:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346325120:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826359.792391:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826359.792448:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346325248:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826359.792473:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:0.0:1787826359.792566:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979088 00000004:00080000:0.0:1787826359.792640:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f78434e4f00 with version 1839 00000020:00080000:0.0:1787826359.792654:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979088 cancel update log [0x240000404:0x2:0x0].945: rc = 0 00000020:00080000:0.0:1787826359.792734:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979088 cancel update log [0x200000bd0:0x2:0x0].942: rc = 0 00000020:00080000:0.0:1787826359.792745:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979088 00000100:00100000:0.0:1787826359.792762:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346325120 00000100:00100000:0.0:1787826359.792771:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346325120 00000100:00100000:0.0:1787826359.792827:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346325248 00000100:00100000:0.0:1787826359.792830:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346325248 00000100:00100000:0.0:1787826359.792856:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11200 00000100:00100000:0.0:1787826359.792866:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346325120:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826359.792881:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826359.792902:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c1ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19b:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c1a7 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.792916:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x4ec94fd6 (ffff9f784b4a5200) 00010000:00010000:0.0:1787826359.792925:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826359.792964:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3655:x1874663346325120:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 99us (573us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.792971:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11200 00000100:00100000:0.0:1787826359.792988:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11201 00000100:00100000:0.0:1787826359.792995:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3652:x1874663346325248:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826359.793003:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826359.793159:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c1df lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c1d1 expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.793174:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f784abfe200) 00010000:00010000:0.0:1787826359.793184:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826359.793210:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346325248:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 216us (737us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826359.793217:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11201 00000100:00100000:0.0:1787826359.793243:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e760a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346325376:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1787826359.793279:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826359.793324:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346325376 00000100:00100000:0.0:1787826359.793329:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346325376 00000100:00100000:0.0:1787826359.793370:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30882 00000100:00100000:0.0:1787826359.793378:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346325376:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1787826359.793408:0:35352:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1315 count 1 00010000:00010000:0.0:1787826359.793432:0:35352:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.793456:0:35352:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.793465:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=1s v=(1 0) 00010000:00010000:0.0:1787826359.793470:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=0s v=(0 1) 00010000:00010000:0.0:1787826359.793475:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.793504:0:35352:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 5 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.793523:0:35352:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 5 pid: 24218 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.793537:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346325504:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826359.794226:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4c00 going to sleep for 11 seconds 00000100:00080000:0.0:1787826359.794245:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787826359.795199:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.795217:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.795228:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.795236:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826359.795241:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.795250:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.795268:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785cc600) 00010000:00010000:3.0:1787826359.795272:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826359.795277:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.795283:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c065 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826359.796023:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346325248:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826359.796068:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346325120:0@lo:103:jbd2/dm-0-8.0 00000004:00080000:0.0:1787826359.796120:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1839 rpc_version 1839 00000020:00100000:0.0:1787826359.796160:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826359.796170:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7847704e00 x1874663346325760/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826359.796195:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346325760:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826359.796261:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826359.796294:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e43c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826359.796327:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346325760 00000100:00100000:0.0:1787826359.796336:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346325760 00000100:00100000:0.0:1787826359.796390:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24089 00000100:00100000:0.0:1787826359.796401:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346325760:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826359.797099:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346325760:12345-0@lo:1000:osp_up1-0.0 Request processed in 695us (833us total) trans 4294985474 rc 0/0 00000100:00100000:0.0:1787826359.797138:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24089 00000100:00100000:0.0:1787826359.797204:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346325760:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826359.798773:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826359.799416:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332675840 00000100:00100000:3.0:1787826359.799430:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332675840 00010000:00010000:3.0:1787826359.799482:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 6 pid: 24218 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826359.799540:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22053 00000100:00100000:3.0:1787826359.799553:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332675840:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826359.799571:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826359.799590:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 6 pid: 24218 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.799613:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 6 pid: 24218 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.799644:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=0s v=(0 1) 00010000:00010000:3.0:1787826359.799650:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x523:0x0].0x0 since=0s v=(0 1) 00010000:00010000:3.0:1787826359.799657:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.799669:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826359.799695:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826359.799763:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332675840:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 211us (999us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826359.799771:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22053 00010000:00010000:2.0:1787826359.804018:0:35352:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039f02c1e/0xe21309aa81f4c1bc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801abc expref: 5 pid: 24218 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826359.804049:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346325504:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826359.804085:0:35352:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.804098:0:35352:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.804132:0:35352:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.804146:0:35352:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.804154:0:35352:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826359.804184:0:35352:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826359.804469:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346325376:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11088us (11186us total) trans 4294976171 rc 0/0 00000100:00100000:2.0:1787826359.804481:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30882 00000004:00080000:2.0:1787826359.804550:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784e760a80/2, rc 0, transno 4294976171 00000100:00100000:2.0:1787826359.804557:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e760a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346325376:0@lo:6:osp-syn-1-0.0 00010000:00010000:0.0:1787826359.804841:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826359.804850:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.804861:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826359.804865:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.804875:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.804897:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.804904:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000844834c3/0xe21309aa81f4c1f4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826359.804927:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826359.804929:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826359.804931:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826360.494350:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826360.542209:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826360.542265:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.542276:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332675968 00000100:00100000:3.0:1787826360.542288:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332675968 00000100:00100000:0.0:1787826360.542300:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826360.542339:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332676224 00000100:00100000:0.0:1787826360.542344:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332676224 00000100:00100000:3.0:1787826360.542360:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2240 00000100:00100000:3.0:1787826360.542374:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784331dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332675968:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826360.542390:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332676096 00000100:00100000:0.0:1787826360.542394:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332676096 00000100:00100000:0.0:1787826360.542419:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14374 00000100:00100000:0.0:1787826360.542429:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332676224:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826360.542457:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784331dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332675968:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 86us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.542469:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2240 00000100:00100000:0.0:1787826360.542470:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332676224:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 41us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.542481:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14374 00000100:00100000:0.0:1787826360.542499:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14375 00000100:00100000:0.0:1787826360.542508:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2395:x1874663332676096:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826360.542541:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2395:x1874663332676096:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 35us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.542548:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14375 00000100:00080000:0.0:1787826360.542568:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787826360.542572:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826360.577566:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.578480:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332676480 00000100:00100000:3.0:1787826360.578491:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332676480 00000100:00100000:3.0:1787826360.578557:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8879 00000100:00100000:3.0:1787826360.578569:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332676480:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826360.578641:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332676480:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 75us (1085us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.578648:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8879 00000100:00100000:3.0:1787826360.579693:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.579977:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.580014:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.580033:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.580052:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826360.580070:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332676608 00000100:00100000:0.0:1787826360.580080:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332676608 00000100:00100000:0.0:1787826360.580147:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332676736 00000100:00100000:0.0:1787826360.580150:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332676736 00000100:00100000:0.0:1787826360.580169:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14376 00000100:00100000:0.0:1787826360.580178:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332676608:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826360.580242:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848302680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332676608:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 65us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.580251:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14376 00000100:00100000:0.0:1787826360.580269:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14377 00000100:00100000:0.0:1787826360.580276:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332676736:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826360.580302:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332676736:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.580325:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14377 00000100:00100000:0.0:1787826360.580351:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332676864 00000100:00100000:0.0:1787826360.580355:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332676864 00000100:00100000:0.0:1787826360.580527:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332676992 00000100:00100000:0.0:1787826360.580532:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332676992 00000100:00100000:0.0:1787826360.580553:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332676352 00000100:00100000:0.0:1787826360.580556:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332676352 00000100:00100000:2.0:1787826360.580565:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8880 00000100:00100000:0.0:1787826360.580585:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8881 00000100:00100000:0.0:1787826360.580592:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332676992:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826360.580620:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332676992:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (586us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.580626:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8881 00000100:00100000:2.0:1787826360.580634:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332676864:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826360.580642:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30883 00000100:00100000:0.0:1787826360.580651:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2392:x1874663332676352:12345-192.168.203.56@tcp:9:kworker.0 00000100:00100000:2.0:1787826360.580672:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332676864:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 97us (656us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.580679:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8880 00000020:00000080:0.0:1787826360.580936:0:17826:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xe21309aa81f466ae 00000100:00080000:0.0:1787826360.580948:0:17826:0:(import.c:65:import_set_state_nolock()) ffff9f7844125800 : changing import state from FULL to CLOSED 00010000:00010000:2.0:1787826360.580992:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000004:0.0:1787826360.581008:0:17826:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294973861 00000100:00100000:0.0:1787826360.581171:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+2:2392:x1874663332676352:12345-192.168.203.56@tcp:9:kworker.0 Request processed in 521us (1117us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.581178:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30883 00000100:00080000:0.0:1787826360.581198:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00100000:1.0:1787826360.584359:0:8438:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:1.0:1787826360.584368:0:8438:0:(genops.c:885:class_export_put()) final put ffff9f7844125000/398e0287-d3c7-44dd-b1d9-6b85f698e615 00000001:00080000:1.0:1787826360.584400:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973861 is committed 00000001:00080000:1.0:1787826360.584413:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973860 is committed 00000020:00000080:2.0:1787826360.584456:0:128839:0:(genops.c:844:class_export_destroy()) destroying export ffff9f7844125000/398e0287-d3c7-44dd-b1d9-6b85f698e615 for lustre-OST0000 00000020:00000080:2.0:1787826360.584518:0:128839:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9f7844125800 for lustre-OST0000 00000100:00100000:0.0:1787826360.612765:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826360.614685:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332677120 00000100:00100000:1.0:1787826360.614697:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332677120 00000100:00100000:1.0:1787826360.614773:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68655 00000100:00100000:1.0:1787826360.614785:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226473:x1874663332677120:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826360.614805:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826360.614829:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1fb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801aca expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.614905:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4c202 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.614919:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4c202 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.614934:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4c202 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826360.615046:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5a:0x0]: returning size 4096 00000004:00200000:1.0:1787826360.615052:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787826360.615073:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4c202 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.615124:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c1fb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aca expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.615146:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4c202 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aca expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.615155:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:1.0:1787826360.615159:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826360.615163:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826360.615238:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226473:x1874663332677120:12345-192.168.203.56@tcp:101:rm.0 Request processed in 456us (2483us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826360.615246:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68655 00000100:00100000:0.0:1787826360.624415:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826360.624604:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332677248 00000100:00100000:1.0:1787826360.624614:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332677248 00000100:00100000:1.0:1787826360.624673:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68656 00000100:00100000:1.0:1787826360.624684:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226473:x1874663332677248:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826360.624703:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826360.624727:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c209 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801ad1 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.624794:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.624806:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.624816:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826360.624859:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19b:0x0]: returning size 4096 00000004:00200000:1.0:1787826360.624864:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826360.624898:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c209 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.624914:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.624922:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:1.0:1787826360.624926:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826360.624929:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826360.625274:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226473:x1874663332677248:12345-192.168.203.56@tcp:101:rm.0 Request processed in 591us (869us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826360.625282:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68656 00000100:00100000:0.0:1787826360.629710:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826360.629851:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332677376 00000100:00100000:1.0:1787826360.629863:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332677376 00000100:00100000:1.0:1787826360.629928:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68657 00000100:00100000:1.0:1787826360.629939:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226473:x1874663332677376:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826360.629961:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826360.629984:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c217 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801ad8 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.630045:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.630060:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.630073:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826360.630148:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x199:0x0]: returning size 4096 00000004:00200000:1.0:1787826360.630154:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826360.630190:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c217 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.630207:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.630215:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:1.0:1787826360.630218:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826360.630222:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826360.630278:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226473:x1874663332677376:12345-192.168.203.56@tcp:101:rm.0 Request processed in 341us (580us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826360.630285:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68657 00000100:00100000:0.0:1787826360.633244:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826360.634551:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332677632 00000100:00100000:2.0:1787826360.634563:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332677632 00000100:00100000:2.0:1787826360.634624:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68658 00000100:00100000:2.0:1787826360.634636:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226473:x1874663332677632:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826360.634655:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826360.634680:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c225 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801adf expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826360.634944:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5a:0x0]: returning size 4096 00010000:00010000:2.0:1787826360.635047:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c225 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801adf expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.635067:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f784abfec00) 00010000:00010000:2.0:1787826360.635070:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826360.635074:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c225 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801adf expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.635092:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826360.635523:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226473:x1874663332677632:12345-192.168.203.56@tcp:101:rm.0 Request processed in 888us (2287us total) trans 4294985475 rc 301/301 00000100:00100000:2.0:1787826360.635531:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68658 00000100:00100000:0.0:1787826360.641248:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826360.641391:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332677760 00000100:00100000:2.0:1787826360.641401:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332677760 00000100:00100000:2.0:1787826360.641462:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9390 00000100:00100000:2.0:1787826360.641475:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7943030050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226473:x1874663332677760:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:2.0:1787826360.655226:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7943030050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226473:x1874663332677760:12345-192.168.203.56@tcp:37:rm.0 Request processed in 13745us (13976us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.655241:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9390 00000100:00100000:0.0:1787826360.659748:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.659830:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332677888 00000100:00100000:3.0:1787826360.659840:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332677888 00000100:00100000:3.0:1787826360.659899:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9391 00000100:00100000:3.0:1787826360.659910:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226473:x1874663332677888:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:3.0:1787826360.666925:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226473:x1874663332677888:12345-192.168.203.56@tcp:37:rm.0 Request processed in 7005us (7176us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.666940:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9391 00000100:00100000:2.0:1787826360.671892:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826360.671998:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332678144 00000100:00100000:0.0:1787826360.672007:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332678144 00000100:00100000:0.0:1787826360.672062:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9392 00000100:00100000:0.0:1787826360.672071:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226473:x1874663332678144:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:0.0:1787826360.672523:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1228, 344 off1121328 00000040:00080000:0.0:1787826360.674080:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1546, 424 off1565184 00000004:00080000:0.0:1787826360.674098:0:17601:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78434e4600 with version 6885 00000004:00080000:2.0:1787826360.674185:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6885 rpc_version 6885 00000020:00100000:2.0:1787826360.674219:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826360.674230:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7847704380 x1874663346326016/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826360.674264:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346326016:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826360.674344:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826360.674389:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7946867f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826360.674551:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346326016 00000100:00100000:0.0:1787826360.674558:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346326016 00000100:00100000:0.0:1787826360.674621:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24090 00000100:00100000:0.0:1787826360.674630:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346326016:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826360.674830:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346326016:12345-0@lo:1000:osp_up0-1.0 Request processed in 200us (486us total) trans 8589957477 rc 0/0 00000100:00100000:0.0:1787826360.674838:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24090 00000100:00100000:3.0:1787826360.674907:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957474 gen 2 00000004:00080000:3.0:1787826360.674912:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957474 committed transno 8589957474 00000100:00100000:3.0:1787826360.674933:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f7847704380 x1874663346326016/t8589957477(8589957477) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826376 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826360.674960:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346326016:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826360.675185:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226473:x1874663332678144:12345-192.168.203.56@tcp:35:rm.0 Request processed in 3113us (3299us total) trans 4294985476 rc 0/0 00000100:00100000:0.0:1787826360.675196:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9392 00000100:00100000:0.0:1787826360.680764:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826360.680803:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332678400 00000100:00100000:0.0:1787826360.680811:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332678400 00000100:00100000:0.0:1787826360.680870:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68659 00000100:00100000:0.0:1787826360.680881:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226473:x1874663332678400:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826360.680911:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826360.680932:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c233 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801ae6 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.681087:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c23a lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.681100:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c23a lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.681133:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c23a lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826360.681145:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5a:0x0]: returning size 4096 00010000:00010000:0.0:1787826360.681238:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c233 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ae6 expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.681259:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c23a lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ae6 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.681268:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:0.0:1787826360.681271:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787826360.681275:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826360.681339:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226473:x1874663332678400:12345-192.168.203.56@tcp:101:rm.0 Request processed in 460us (581us total) trans 4294985477 rc 0/0 00000100:00100000:0.0:1787826360.681346:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68659 00000100:00100000:0.0:1787826360.691357:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.691972:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332678528 00000100:00100000:3.0:1787826360.691983:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332678528 00000100:00100000:3.0:1787826360.692102:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68660 00000100:00100000:3.0:1787826360.692138:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226473:x1874663332678528:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:3.0:1787826360.692196:0:8752:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:3.0:1787826360.692269:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226473:x1874663332678528:12345-192.168.203.56@tcp:41:rm.0 Request processed in 134us (917us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.692277:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68660 00000100:00100000:0.0:1787826360.702285:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.702344:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332678656 00000100:00100000:3.0:1787826360.702355:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332678656 00000100:00100000:3.0:1787826360.702488:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68661 00000100:00100000:3.0:1787826360.702500:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226473:x1874663332678656:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826360.702518:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826360.702558:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c248 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801aed expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702632:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702646:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702657:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702673:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702679:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702687:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702744:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c25d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702752:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c25d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702758:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c25d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826360.702781:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c25d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702797:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702804:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702810:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702817:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702824:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826360.702838:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702846:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702858:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:3.0:1787826360.702862:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826360.702867:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 3/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702873:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c24f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702886:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702892:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702897:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702902:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826360.702905:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702912:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702919:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785cc600) 00010000:00010000:3.0:1787826360.702921:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826360.702924:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702930:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c256 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702962:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c248 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aed expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702976:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c25d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aed expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.702984:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:3.0:1787826360.702986:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826360.702988:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69c00, rc 0) 00000100:00100000:3.0:1787826360.703059:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226473:x1874663332678656:12345-192.168.203.56@tcp:101:rm.0 Request processed in 561us (781us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.703067:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68661 00000100:00100000:0.0:1787826360.708961:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826360.710020:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332678784 00000100:00100000:2.0:1787826360.710031:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332678784 00000100:00100000:2.0:1787826360.710101:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68662 00000100:00100000:2.0:1787826360.710130:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226473:x1874663332678784:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826360.710155:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826360.710180:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4c264 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801af4 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.710246:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c26b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.710259:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c26b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.710270:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c26b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826360.710332:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5b:0x0]: returning size 6 00000004:00200000:2.0:1787826360.710338:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:2.0:1787826360.710376:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efc00/0xe21309aa81f4c264 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801af4 expref: 14 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.710394:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c26b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801af4 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.710409:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f784abfec00) 00010000:00010000:2.0:1787826360.710412:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826360.710416:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826360.710487:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226473:x1874663332678784:12345-192.168.203.56@tcp:101:rm.0 Request processed in 360us (1533us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.710495:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68662 00000100:00100000:1.0:1787826360.721600:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826360.721868:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826360.721899:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332678912 00000100:00100000:2.0:1787826360.721909:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332678912 00000100:00100000:2.0:1787826360.721971:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11202 00000100:00100000:2.0:1787826360.721981:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2396:x1874663332678912:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826360.721996:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826360.722035:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c25d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aed expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.722054:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f79785cce00) 00010000:00010000:2.0:1787826360.722065:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826360.722249:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2396:x1874663332678912:12345-192.168.203.56@tcp:103:rm.0 Request processed in 269us (659us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.722257:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11202 00000100:00100000:3.0:1787826360.722421:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332679168 00000100:00100000:3.0:1787826360.722427:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332679168 00000100:00100000:3.0:1787826360.724083:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68663 00000100:00100000:3.0:1787826360.724098:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226473:x1874663332679168:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826360.724179:0:24703:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.724204:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee600/0xe21309aa81f4c26b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801af4 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.724223:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f784abfec00) 00010000:00010000:3.0:1787826360.724236:0:24703:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826360.724286:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.724295:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.724316:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.724343:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326144:0@lo:101:mdt00_007.0 00000100:00100000:3.0:1787826360.724405:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.724442:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826360.724465:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346326144 00000100:00100000:3.0:1787826360.724472:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346326144 00000100:00100000:3.0:1787826360.724503:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24091 00000100:00100000:3.0:1787826360.724509:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:24703:x1874663346326144:12345-0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826360.724524:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826360.724538:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c272 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.724559:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.724570:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c272 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.724608:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.724623:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 9 pid: 35080 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.724635:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851884a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346326272:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826360.724701:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826360.729485:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851884a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346326272:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826360.729525:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c272 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.729539:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:3.0:1787826360.729542:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826360.729547:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c272 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.729559:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826360.729690:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:24703:x1874663346326144:12345-0@lo:101:mdt00_007.0 Request processed in 5179us (5284us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.729700:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24091 00000100:00100000:2.0:1787826360.729803:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326144:0@lo:101:mdt00_007.0 00010000:00010000:2.0:1787826360.729825:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.729836:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.731533:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.731867:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332679296 00000100:00100000:3.0:1787826360.731877:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332679296 00010000:00010000:3.0:1787826360.732064:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 10 pid: 35080 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.732130:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22054 00000100:00100000:3.0:1787826360.732141:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7947dfc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332679296:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826360.732156:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.732163:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 10 pid: 35080 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732176:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 10 pid: 35080 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732200:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c21e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x199:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad8 expref: 10 pid: 35080 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732215:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:3.0:1787826360.732222:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c272 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732232:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c272 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732242:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c272 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732276:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c272 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.732290:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851884a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346326400:0@lo:105: 00000100:00100000:3.0:1787826360.732357:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.732406:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826360.732429:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346326400 00000100:00100000:3.0:1787826360.732433:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346326400 00000100:00100000:3.0:1787826360.732466:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7918 00000100:00100000:3.0:1787826360.732474:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346326400:12345-0@lo:105: 00010000:00010000:3.0:1787826360.732486:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732496:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732509:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732517:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732525:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732534:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.732556:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68600) 00000100:00100000:3.0:1787826360.733096:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346326400:12345-0@lo:105: Request processed in 620us (737us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.733128:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7918 00010000:00010000:3.0:1787826360.733164:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.733175:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826360.733203:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x199:0x0] 00010000:00010000:3.0:1787826360.733237:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.733246:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.733257:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.733271:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326528:0@lo:101:mdt00_007.0 00000100:00100000:3.0:1787826360.733315:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.733341:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826360.733361:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346326528 00000100:00100000:3.0:1787826360.733365:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346326528 00000100:00100000:3.0:1787826360.733412:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24092 00000100:00100000:3.0:1787826360.733422:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:24703:x1874663346326528:12345-0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826360.733438:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826360.733451:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c287 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c280 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.733473:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c287 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c280 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.733485:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c287 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c280 expref: 10 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.733493:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826360.733496:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826360.733499:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826360.733532:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346326528:12345-0@lo:101:mdt00_007.0 Request processed in 112us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.733538:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24092 00000100:00100000:3.0:1787826360.733593:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326528:0@lo:101:mdt00_007.0 00010000:00010000:3.0:1787826360.733611:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c287 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.733621:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c287 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826360.733639:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x199:0x0] 00000100:00100000:1.0:1787826360.733657:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851884a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346326400:0@lo:105: 00010000:00010000:1.0:1787826360.733677:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826360.733680:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326656:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826360.733718:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.733739:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826360.733757:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346326656 00000100:00100000:3.0:1787826360.733761:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346326656 00000100:00100000:3.0:1787826360.733794:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24093 00000100:00100000:3.0:1787826360.733803:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346326656:12345-0@lo:1000:mdt00_007.0 00000100:00100000:1.0:1787826360.733891:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7947dfc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332679296:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1589us (2203us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.733898:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346326656:12345-0@lo:1000:mdt00_007.0 Request processed in 96us (179us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826360.733907:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22054 00000100:00100000:3.0:1787826360.733910:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24093 00000100:00100000:3.0:1787826360.734048:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326656:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826360.734098:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326784:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826360.734142:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.734158:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826360.734170:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346326784 00000100:00100000:3.0:1787826360.734174:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346326784 00000100:00100000:3.0:1787826360.734194:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24094 00000100:00100000:3.0:1787826360.734199:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346326784:12345-0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826360.734260:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346326784:12345-0@lo:1000:mdt00_007.0 Request processed in 59us (116us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.734267:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24094 00000100:00100000:3.0:1787826360.734297:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326784:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826360.734355:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326912:0@lo:1000:mdt00_007.0 00000100:00100000:3.0:1787826360.734384:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.734413:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826360.734488:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346326912 00000100:00100000:0.0:1787826360.734494:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346326912 00000100:00100000:0.0:1787826360.734539:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24095 00000100:00100000:0.0:1787826360.734549:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346326912:12345-0@lo:1000:mdt00_007.0 00000100:00100000:0.0:1787826360.734647:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346326912:12345-0@lo:1000:mdt00_007.0 Request processed in 99us (262us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826360.734655:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24095 00000100:00100000:0.0:1787826360.734693:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346326912:0@lo:1000:mdt00_007.0 00010000:00010000:0.0:1787826360.734931:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.734942:0:24703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.734952:0:24703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.734967:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346327040:0@lo:101:mdt00_007.0 00000100:00100000:0.0:1787826360.735000:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826360.735025:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826360.735041:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346327040 00000100:00100000:1.0:1787826360.735047:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346327040 00000100:00100000:1.0:1787826360.735086:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24096 00000100:00100000:1.0:1787826360.735093:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:24703:x1874663346327040:12345-0@lo:101:mdt00_007.0 00010000:00010000:1.0:1787826360.735102:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826360.735142:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c295 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c28e expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.735163:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c295 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c28e expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.735174:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c295 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c28e expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.735183:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:1.0:1787826360.735186:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826360.735188:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826360.735219:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:24703:x1874663346327040:12345-0@lo:101:mdt00_007.0 Request processed in 127us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826360.735224:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24096 00000100:00100000:0.0:1787826360.735319:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-mdtlov_UUID:24703:1874663346327040:0@lo:101:mdt00_007.0 00010000:00010000:0.0:1787826360.735335:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.735345:0:24703:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.735371:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.735379:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.735388:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787826360.735740:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].230, 64 off22912 00000004:00080000:0.0:1787826360.735754:0:24703:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8172947648064ns, next at 8173947645707ns, rc = 0 00000040:00080000:0.0:1787826360.735785:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1229, 384 off1121712 00000040:00080000:0.0:1787826360.737417:0:24703:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1547, 464 off1565648 00000004:00080000:0.0:1787826360.737432:0:24703:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78434e4c00 with version 6886 00000004:00080000:3.0:1787826360.737546:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6886 rpc_version 6886 00000020:00100000:3.0:1787826360.737643:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826360.737654:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7848cfd180 x1874663346327296/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826360.737678:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfd180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346327296:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826360.737742:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.737779:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826360.737935:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346327296 00000100:00100000:0.0:1787826360.737942:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346327296 00000100:00100000:0.0:1787826360.737988:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24097 00000100:00100000:0.0:1787826360.737997:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346327296:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826360.738340:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346327296:12345-0@lo:1000:osp_up0-1.0 Request processed in 344us (598us total) trans 8589957478 rc 0/0 00000100:00100000:0.0:1787826360.738349:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24097 00000100:00100000:2.0:1787826360.738951:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfd180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346327296:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787826360.739003:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739023:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739033:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739042:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739049:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739056:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739077:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826360.739098:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.739101:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826360.739274:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739806:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739814:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.739898:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826360.739903:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826360.739905:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739911:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.739917:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739921:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.739948:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:2.0:1787826360.739953:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826360.739957:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:2.0:1787826360.739958:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739962:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826360.739967:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.739969:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c2a3 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e5b:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.739983:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c29c lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.739991:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826360.739995:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826360.739999:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826360.740004:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.740014:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.740022:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.740041:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826360.740056:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x199:0x0] 00010000:00010000:2.0:1787826360.740059:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826360.740065:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.740067:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.740083:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.740085:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826360.740094:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.740095:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826360.740262:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.740266:0:24703:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787826360.740275:0:24703:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x199:0x0] 00010000:00010000:0.0:1787826360.740285:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c287 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.740293:0:24703:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c287 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.740387:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226473:x1874663332679168:12345-192.168.203.56@tcp:36:rm.0 Request processed in 16290us (18516us total) trans 4294985478 rc 0/0 00000100:00100000:0.0:1787826360.740396:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68663 00010000:00010000:2.0:1787826360.740580:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.740591:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826360.740627:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.740635:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c28e lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c295 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.740651:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826360.740655:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.740660:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826360.740664:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826360.740671:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.740686:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346327424:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826360.740690:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.740701:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826360.740708:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.740721:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c272 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c279 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.740742:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826360.740774:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826360.740781:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e763800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346327552:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826360.740802:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826360.741012:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826360.741014:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826360.741016:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826360.741028:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826360.741044:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346327424 00000100:00100000:3.0:1787826360.741050:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346327424 00000100:00100000:3.0:1787826360.741099:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346327552 00000100:00100000:3.0:1787826360.741125:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346327552 00000100:00100000:3.0:1787826360.741152:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11203 00000100:00100000:3.0:1787826360.741161:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346327424:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826360.741174:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.741192:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c295 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5b:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c28e expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.741204:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5b:0x0].0x0 (ffff9f7845cb1200) 00010000:00010000:3.0:1787826360.741211:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826360.741343:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346327424:12345-0@lo:103:ldlm_bl.0 Request processed in 183us (601us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.741351:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11203 00000100:00100000:3.0:1787826360.741365:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11204 00000100:00100000:3.0:1787826360.741372:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346327552:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826360.741380:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.741391:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c279 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x199:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c272 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.741403:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f7845cb1500) 00010000:00010000:3.0:1787826360.741409:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826360.741430:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346327552:12345-0@lo:103:ldlm_bl.0 Request processed in 59us (627us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.741436:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11204 00000100:00100000:3.0:1787826360.741472:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e763800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346327552:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826360.741481:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346327424:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826360.745754:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826360.745806:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826360.745806:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332679424 00000100:00100000:2.0:1787826360.745817:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332679424 00000100:00100000:2.0:1787826360.745900:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11205 00000100:00100000:2.0:1787826360.745912:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332679424:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826360.745927:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826360.745952:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c0ce lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a6f expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.745971:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:2.0:1787826360.745975:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826360.745979:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332679680 00000100:00100000:3.0:1787826360.745986:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332679680 00000100:00100000:2.0:1787826360.746037:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332679424:12345-192.168.203.56@tcp:103:rm.0 Request processed in 128us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.746044:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11205 00000100:00100000:0.0:1787826360.746776:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9393 00000100:00100000:0.0:1787826360.746789:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226473:x1874663332679680:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:0.0:1787826360.747054:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:226473:x1874663332679680:12345-192.168.203.56@tcp:35:rm.0 Request processed in 267us (1246us total) trans 4294985479 rc 0/0 00000100:00100000:0.0:1787826360.747062:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9393 00000100:00100000:1.0:1787826360.750505:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.750589:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332679936 00000100:00100000:3.0:1787826360.750598:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332679936 00000100:00100000:3.0:1787826360.750665:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68664 00000100:00100000:3.0:1787826360.750676:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:226473:x1874663332679936:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826360.750731:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826360.750757:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43800/0xe21309aa81f4c23a lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ae6 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.750777:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece400/0xe21309aa81f4c202 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aca expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.750790:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef000/0xe21309aa81f4c0dc lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a76 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.750801:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:3.0:1787826360.750805:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826360.750846:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.750855:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.750865:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.750886:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cff100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346327680:0@lo:101:mdt00_011.0 00000100:00100000:3.0:1787826360.750931:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.750971:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826360.751077:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346327680 00000100:00100000:0.0:1787826360.751082:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346327680 00000100:00100000:0.0:1787826360.751142:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24098 00000100:00100000:0.0:1787826360.751150:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346327680:12345-0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826360.751163:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826360.751180:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c2aa expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.751200:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.751209:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.751217:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.751240:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.751253:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 7 pid: 24702 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.751265:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346327808:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826360.751339:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.751350:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 7 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.751358:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346327936:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826360.751389:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826360.756517:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.756609:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332680064 00000100:00100000:3.0:1787826360.756618:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332680064 00010000:00010000:3.0:1787826360.756665:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 8 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.756709:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22055 00000100:00100000:3.0:1787826360.756719:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332680064:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826360.756741:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.756749:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 8 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.756767:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 8 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.756790:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826360.756798:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.756808:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826360.757544:0:8152:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c0b2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a68 expref: 8 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.757566:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346327936:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826360.757579:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346327808:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826360.757609:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.757620:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826360.757623:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826360.757627:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.757635:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:3.0:1787826360.757712:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332680064:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 993us (1202us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.757722:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22055 00000100:00100000:2.0:1787826360.757824:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826360.757911:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332680192 00000100:00100000:3.0:1787826360.757917:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332680192 00010000:00010000:3.0:1787826360.757939:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 8 pid: 24702 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.757972:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22056 00000100:00100000:3.0:1787826360.757980:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332680192:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826360.757989:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.757995:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 8 pid: 24702 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.758006:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 8 pid: 24702 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.758063:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c11b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a92 expref: 8 pid: 24702 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.758077:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826360.758082:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.758091:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.758100:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.758161:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826360.758176:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346328064:0@lo:105: 00000100:00100000:3.0:1787826360.758320:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.758365:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826360.758477:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346328064 00000100:00100000:2.0:1787826360.758482:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346328064 00000100:00100000:2.0:1787826360.758521:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7919 00000100:00100000:2.0:1787826360.758529:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346328064:12345-0@lo:105: 00010000:00010000:2.0:1787826360.758540:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.758550:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.758558:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.758570:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.758589:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.758598:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.758611:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68c00) 00000100:00100000:2.0:1787826360.758663:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6513:x1874663346328064:12345-0@lo:105: Request processed in 134us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.758670:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7919 00000100:00100000:0.0:1787826360.758980:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346327680:12345-0@lo:101:mdt00_011.0 Request processed in 7829us (8046us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.758987:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24098 00000100:00100000:1.0:1787826360.760428:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cff100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346327680:0@lo:101:mdt00_011.0 00010000:00010000:1.0:1787826360.760465:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826360.760497:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826360.760542:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.760550:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.760560:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.760578:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346328192:0@lo:101:mdt00_011.0 00000100:00100000:3.0:1787826360.760931:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6513:1874663346328064:0@lo:105: 00000100:00100000:1.0:1787826360.760932:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826360.760952:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826360.760973:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d0673c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826360.761008:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332680192:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3026us (3183us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.761018:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22056 00000100:00100000:0.0:1787826360.761046:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346328192 00000100:00100000:0.0:1787826360.761053:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346328192 00000100:00100000:0.0:1787826360.761096:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24099 00000100:00100000:0.0:1787826360.761121:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346328192:12345-0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826360.761135:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826360.761151:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c2bf lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c2b8 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.761172:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c2bf lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c2b8 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.761184:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c2bf lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2b8 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.761192:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a200) 00010000:00010000:0.0:1787826360.761195:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826360.761197:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826360.762805:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346328192:12345-0@lo:101:mdt00_011.0 Request processed in 1696us (1868us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.762816:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24099 00000100:00100000:3.0:1787826360.763347:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346328192:0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826360.763880:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.763904:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826360.763928:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826360.763981:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346328320:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826360.764036:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.764064:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826360.765614:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346328320 00000100:00100000:2.0:1787826360.765628:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346328320 00000100:00100000:2.0:1787826360.765690:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24100 00000100:00100000:2.0:1787826360.765712:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346328320:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826360.765829:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346328320:12345-0@lo:1000:mdt00_011.0 Request processed in 121us (1792us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.765838:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24100 00000100:00100000:1.0:1787826360.765930:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851887100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346328320:0@lo:1000:mdt00_011.0 00000100:00100000:1.0:1787826360.765995:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346328448:0@lo:1000:mdt00_011.0 00000100:00100000:1.0:1787826360.766030:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826360.766060:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826360.766075:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346328448 00000100:00100000:2.0:1787826360.766081:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346328448 00000100:00100000:2.0:1787826360.766141:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24101 00000100:00100000:2.0:1787826360.766149:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346328448:12345-0@lo:1000:mdt00_011.0 00000100:00100000:2.0:1787826360.766241:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346328448:12345-0@lo:1000:mdt00_011.0 Request processed in 92us (209us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826360.766248:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24101 00000100:00100000:1.0:1787826360.766273:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346328448:0@lo:1000:mdt00_011.0 00010000:00010000:1.0:1787826360.766367:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.766435:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.766445:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.766458:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346328576:0@lo:101:mdt00_011.0 00000100:00100000:1.0:1787826360.766482:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826360.766504:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826360.766530:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346328576 00000100:00100000:0.0:1787826360.766535:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346328576 00000100:00100000:0.0:1787826360.766569:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24102 00000100:00100000:0.0:1787826360.766576:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346328576:12345-0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826360.766585:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826360.766601:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c2c6 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.766627:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.766647:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c2c6 expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.766673:0:8152:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.766686:0:8152:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 6 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.766706:0:8152:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346328704:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826360.766771:0:8152:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826360.769334:0:8152:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:8152:1874663346328704:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826360.769373:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2c6 expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.769386:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:0.0:1787826360.769390:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826360.769394:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2c6 expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.769404:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43400, rc 0) 00000100:00100000:0.0:1787826360.769518:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346328576:12345-0@lo:101:mdt00_011.0 Request processed in 2941us (3033us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.769527:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24102 00000100:00100000:2.0:1787826360.770621:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826360.770730:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332680320 00000100:00100000:0.0:1787826360.770738:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332680320 00010000:00010000:0.0:1787826360.770772:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 7 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.770807:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22057 00000100:00100000:0.0:1787826360.770816:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332680320:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826360.770829:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826360.770835:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 7 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.770848:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 7 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.770871:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c137 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a99 expref: 7 pid: 8752 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.770902:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:0.0:1787826360.770908:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2c6 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.770918:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c2c6 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.770928:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c2c6 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.770957:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2c6 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.770973:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346328832:0@lo:105: 00000100:00100000:0.0:1787826360.771015:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826360.771053:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826360.771067:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346328832 00000100:00100000:3.0:1787826360.771073:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346328832 00000100:00100000:3.0:1787826360.771129:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7920 00000100:00100000:3.0:1787826360.771136:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346328832:12345-0@lo:105: 00010000:00010000:3.0:1787826360.771148:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.771159:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.771167:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.771175:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.771182:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.771191:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.771205:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecee00) 00000100:00100000:3.0:1787826360.771239:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346328832:12345-0@lo:105: Request processed in 104us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.771246:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7920 00000100:00100000:0.0:1787826360.771341:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346328832:0@lo:105: 00010000:00010000:0.0:1787826360.771355:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826360.771442:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332680320:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 626us (826us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.771449:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22057 00000100:00100000:1.0:1787826360.772545:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346328576:0@lo:101:mdt00_011.0 00010000:00010000:1.0:1787826360.772571:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.772610:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.772621:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.772631:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.772662:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.772673:0:35094:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.772686:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 4/0,1 mode: --/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.772724:0:35094:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.772737:0:35094:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 11 pid: 35095 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.772750:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cff100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001_UUID:35094:1874663346328960:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826360.772825:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826360.774644:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cff100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001_UUID:35094:1874663346328960:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826360.774673:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826360.775816:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826360.775921:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332680448 00000100:00100000:2.0:1787826360.775930:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332680448 00010000:00010000:2.0:1787826360.775970:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 12 pid: 35095 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826360.776002:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22058 00000100:00100000:2.0:1787826360.776011:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c82050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332680448:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826360.776027:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826360.776034:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 12 pid: 35095 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776046:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 12 pid: 35095 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776068:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c210 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ad1 expref: 12 pid: 35095 timeout: 8272 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776083:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:2.0:1787826360.776089:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776099:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776137:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776157:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826360.776207:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c82050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332680448:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 196us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.776214:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22058 00010000:00010000:1.0:1787826360.776328:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.776337:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.776380:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.776390:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.776401:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.776422:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cff100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346329088:0@lo:101:mdt00_011.0 00000100:00100000:1.0:1787826360.776472:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826360.776503:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826360.776603:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346329088 00000100:00100000:2.0:1787826360.776608:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346329088 00000100:00100000:2.0:1787826360.776654:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24103 00000100:00100000:2.0:1787826360.776662:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346329088:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826360.776677:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826360.776691:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c2e9 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x199:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c2e2 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776721:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c2e9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c2e2 expref: 14 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776733:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c2e9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2e2 expref: 14 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826360.776742:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f784abfea00) 00010000:00010000:2.0:1787826360.776745:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826360.776747:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826360.776787:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346329088:12345-0@lo:101:mdt00_011.0 Request processed in 125us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.776794:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24103 00000100:00100000:1.0:1787826360.776944:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cff100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346329088:0@lo:101:mdt00_011.0 00010000:00010000:1.0:1787826360.776957:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.776968:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.778069:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346329216:0@lo:403:mdt00_011.0 00000100:00100000:1.0:1787826360.778222:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826360.778280:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d067a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826360.778929:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346329216 00000100:00100000:2.0:1787826360.778937:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346329216 00000100:00100000:2.0:1787826360.779034:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24104 00000100:00100000:2.0:1787826360.779043:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346329216:12345-0@lo:403:mdt00_011.0 00000100:00100000:2.0:1787826360.780793:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:35094:x1874663346329216:12345-0@lo:403:mdt00_011.0 Request processed in 1745us (2565us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.780807:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24104 00000100:00100000:1.0:1787826360.780836:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378e00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346329216:0@lo:403:mdt00_011.0 00000040:00080000:1.0:1787826360.782076:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1230, 952 off1122664 00000040:00080000:1.0:1787826360.782286:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1548, 1032 off1566680 00000004:00080000:1.0:1787826360.782293:0:35094:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d067a80 with version 6887 00000004:00080000:2.0:1787826360.782323:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6887 rpc_version 6887 00000020:00100000:2.0:1787826360.782354:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826360.782363:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7850773100 x1874663346329472/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826360.782518:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850773100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346329472:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826360.782568:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826360.782682:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826360.782966:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346329472 00000100:00100000:1.0:1787826360.782974:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346329472 00000100:00100000:1.0:1787826360.783024:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24105 00000100:00100000:1.0:1787826360.783033:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346329472:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826360.784540:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346329472:12345-0@lo:1000:osp_up0-1.0 Request processed in 1503us (1965us total) trans 8589957479 rc 0/0 00000100:00100000:1.0:1787826360.784551:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24105 00000100:00100000:2.0:1787826360.784801:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850773100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346329472:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787826360.785618:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5a:0x0]: returning size 4096 00010000:00010000:1.0:1787826360.785647:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785657:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785666:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785672:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826360.785677:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785684:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785710:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19b:0x0].0x0 (ffff9f7845cb1b00) 00010000:00010000:1.0:1787826360.785714:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826360.785720:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 2/0,0 mode: --/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785725:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2db lrc: 0/0,0 mode: --/EX res: [0x240000400:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785745:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785754:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785761:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785794:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826360.785803:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x199:0x0] 00010000:00010000:1.0:1787826360.785811:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785817:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785823:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785828:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826360.785831:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785837:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785846:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:1.0:1787826360.785848:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826360.785852:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785859:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c2d4 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785869:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785875:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785883:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785905:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787826360.785917:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826360.785922:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785929:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785936:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785955:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787826360.785959:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826360.785963:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826360.785970:0:35094:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826360.786051:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226473:x1874663332679936:12345-192.168.203.56@tcp:36:rm.0 Request processed in 35373us (35551us total) trans 4294985480 rc 0/0 00000100:00100000:1.0:1787826360.786059:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68664 00010000:00010000:3.0:1787826360.786268:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826360.786275:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786283:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826360.786288:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826360.786296:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786337:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786345:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826360.786381:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786388:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c2e2 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2e9 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786405:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826360.786408:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786415:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826360.786419:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826360.786425:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786452:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786458:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826360.786465:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786472:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecee00/0xe21309aa81f4c2c6 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2cd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786484:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826360.786487:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786494:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826360.786497:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826360.786504:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786525:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786530:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826360.786546:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786554:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68c00/0xe21309aa81f4c2aa lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c2b1 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786563:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826360.786565:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826360.786567:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826360.786583:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346329728:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826360.786630:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.786721:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826360.786729:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346329600:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826360.786751:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.786772:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826360.786780:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346329856:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826360.786803:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826360.786832:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346329728 00000100:00100000:3.0:1787826360.786838:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346329728 00000100:00100000:3.0:1787826360.786879:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346329600 00000100:00100000:3.0:1787826360.786882:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346329600 00000100:00100000:3.0:1787826360.786898:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346329856 00000100:00100000:3.0:1787826360.786901:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346329856 00000100:00100000:3.0:1787826360.786917:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11206 00000100:00100000:3.0:1787826360.786925:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+17:3653:x1874663346329728:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826360.786936:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.786955:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c2cd lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c2c6 expref: 17 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.786968:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7952ca9300) 00010000:00010000:3.0:1787826360.786976:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826360.787002:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+16:3653:x1874663346329728:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.787009:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11206 00000100:00100000:3.0:1787826360.787025:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11207 00000100:00100000:3.0:1787826360.787031:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+15:3653:x1874663346329600:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826360.787038:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.787048:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c2e9 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x199:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c2e2 expref: 15 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.787058:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x0 (ffff9f784abfea00) 00010000:00010000:3.0:1787826360.787065:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826360.787084:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+14:3653:x1874663346329600:12345-0@lo:103:ldlm_bl.0 Request processed in 54us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.787091:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11207 00000100:00100000:3.0:1787826360.787130:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11208 00000100:00100000:3.0:1787826360.787138:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346329856:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826360.787145:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826360.787156:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c2b1 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c2aa expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826360.787171:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826360.787175:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826360.787198:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346329856:12345-0@lo:103:ldlm_bl.0 Request processed in 62us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826360.787203:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11208 00000100:00100000:3.0:1787826360.787235:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346329856:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826360.787265:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346329600:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826360.787272:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346329728:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826360.787493:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826360.787651:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826360.900718:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826360.901409:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332680576 00000100:00100000:2.0:1787826360.901419:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332680576 00000100:00100000:2.0:1787826360.901540:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68665 00000100:00100000:2.0:1787826360.901552:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226481:x1874663332680576:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:2.0:1787826360.901602:0:35093:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:2.0:1787826360.901741:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226481:x1874663332680576:12345-192.168.203.56@tcp:41:stat.0 Request processed in 191us (1032us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826360.901749:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68665 00000100:00100000:1.0:1787826360.989784:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826360.989878:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332680704 00000100:00100000:0.0:1787826360.989888:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332680704 00000100:00100000:0.0:1787826360.990000:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68666 00000100:00100000:0.0:1787826360.990012:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226485:x1874663332680704:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826360.990032:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826360.990053:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c2f0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801afb expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.990158:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.990171:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.990184:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826360.990237:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826360.990241:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826360.990261:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.990286:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c2f0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.990317:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826360.990327:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826360.990330:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826360.990334:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826360.990403:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226485:x1874663332680704:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 394us (629us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826360.990414:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68666 00000100:00100000:0.0:1787826360.993820:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826360.999073:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332680832 00000100:00100000:1.0:1787826360.999089:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332680832 00000100:00100000:1.0:1787826360.999224:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68667 00000100:00100000:1.0:1787826360.999236:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226485:x1874663332680832:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:1.0:1787826360.999257:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826360.999279:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2fe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b02 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.000202:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.000219:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.000231:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.000242:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.000252:0:11629:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c2bf lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c2b8 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.000260:0:11629:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.000295:0:11629:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c2bf lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4c2b8 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826361.000320:0:11629:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346329984:0@lo:104: 00000100:00100000:1.0:1787826361.000388:0:11629:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.000505:0:11629:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78451f4b40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826361.002589:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346329984 00000100:00100000:3.0:1787826361.002601:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346329984 00000100:00100000:3.0:1787826361.002772:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7921 00000100:00100000:3.0:1787826361.002783:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346329984:12345-0@lo:104: 00010000:00010000:3.0:1787826361.002799:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.003263:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826361.003269:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:11629:x1874663346329984:12345-0@lo:104: Request processed in 490us (2881us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.003276:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7921 00000100:00100000:1.0:1787826361.003357:0:11629:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11629:1874663346329984:0@lo:104: 00010000:00010000:1.0:1787826361.003385:0:11629:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.003438:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826361.003446:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.003455:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826361.003524:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826361.003534:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.003569:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.003575:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826361.003622:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346330112:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826361.003789:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826361.003864:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.003873:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece200/0xe21309aa81f4c2b8 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c2bf expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.003895:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826361.003897:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826361.003899:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826361.009829:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346330112 00000100:00100000:0.0:1787826361.009853:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346330112 00010000:00010000:0.0:1787826361.009906:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c2bf lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4c2b8 expref: 12 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826361.009954:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22059 00000100:00100000:0.0:1787826361.009964:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3655:x1874663346330112:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826361.009980:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787826361.018381:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957479 is committed 00010000:00010000:3.0:1787826361.019945:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c2bf lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4c2b8 expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.019973:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a200) 00010000:00010000:3.0:1787826361.019979:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.019988:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.019995:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.020082:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826361.020300:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346330112:12345-0@lo:103:ldlm_bl.0 Request processed in 10336us (16510us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.020323:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22059 00010000:00010000:1.0:1787826361.020392:0:11629:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020401:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020535:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020544:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020550:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020556:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020562:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826361.020566:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020638:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020763:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826361.020766:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826361.020770:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020777:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c305 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020788:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020794:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020799:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020803:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826361.020806:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020811:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020817:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a200) 00010000:00010000:1.0:1787826361.020819:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826361.020821:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020826:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c30c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020869:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2fe lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b02 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020882:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826361.020884:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826361.020887:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c2fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b02 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.020896:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826361.020995:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226485:x1874663332680832:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 21761us (27182us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826361.021002:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68667 00000100:00100000:0.0:1787826361.021151:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346330112:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826361.028488:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826361.028582:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332680960 00000100:00100000:0.0:1787826361.028593:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332680960 00000100:00100000:0.0:1787826361.028664:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68668 00000100:00100000:0.0:1787826361.028675:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226485:x1874663332680960:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826361.028694:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826361.028714:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c313 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b09 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.028781:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.028794:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.028805:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826361.028867:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826361.028872:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826361.028892:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.028919:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c313 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.028935:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 8 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.028943:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826361.028947:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826361.028951:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826361.029014:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226485:x1874663332680960:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 342us (537us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826361.029021:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68668 00000100:00100000:3.0:1787826361.039271:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826361.043299:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332681216 00000100:00100000:1.0:1787826361.043322:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332681216 00000100:00100000:1.0:1787826361.043449:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68669 00000100:00100000:1.0:1787826361.043461:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226485:x1874663332681216:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:1.0:1787826361.043583:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848300380 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346330240:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826361.043660:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.043739:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318240 going to sleep for 11 seconds 00000100:00080000:0.0:1787826361.044001:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826361.044034:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346330240 00000100:00100000:3.0:1787826361.044089:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346330240 00000100:00100000:3.0:1787826361.044173:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24106 00000100:00100000:3.0:1787826361.044181:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346330240:12345-0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.044260:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346330240:12345-0@lo:1000:mdt00_013.0 Request processed in 80us (599us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.044267:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24106 00000100:00100000:1.0:1787826361.044340:0:35098:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957479 gen 2 00000004:00080000:1.0:1787826361.044346:0:35098:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957477 committed transno 8589957479 00000004:00080000:1.0:1787826361.044371:0:35098:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957478 committed transno 8589957479 00000004:00080000:1.0:1787826361.044384:0:35098:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957479 committed transno 8589957479 00000100:00100000:1.0:1787826361.044400:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848300380 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346330240:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826361.044447:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848300380 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346330368:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826361.044473:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.044741:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.044811:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346330368 00000100:00100000:2.0:1787826361.044816:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346330368 00000100:00100000:2.0:1787826361.044865:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24107 00000100:00100000:2.0:1787826361.044874:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346330368:12345-0@lo:1000:mdt00_013.0 00000100:00100000:2.0:1787826361.044982:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346330368:12345-0@lo:1000:mdt00_013.0 Request processed in 109us (508us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826361.044989:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24107 00000100:00100000:2.0:1787826361.045018:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848300380 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346330368:0@lo:1000:mdt00_013.0 00010000:00010000:2.0:1787826361.045075:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.045085:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.045098:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826361.045131:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850772680 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346330496:0@lo:101:mdt00_013.0 00000100:00100000:2.0:1787826361.045157:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826361.045174:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.045185:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346330496 00000100:00100000:2.0:1787826361.045188:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346330496 00000100:00100000:2.0:1787826361.045205:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24108 00000100:00100000:2.0:1787826361.045210:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346330496:12345-0@lo:101:mdt00_013.0 00010000:00010000:2.0:1787826361.045218:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826361.045231:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c321 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.045255:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.045264:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.045273:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.045295:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 7 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.045322:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 7 pid: 6527 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826361.045334:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850773100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346330624:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826361.045405:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.045414:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 6 pid: 6528 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826361.045422:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346330752:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826361.045444:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f4540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.049415:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826361.049449:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826361.049993:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850773100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346330624:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826361.050007:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346330752:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826361.050043:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.050056:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826361.050059:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826361.050063:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.050074:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:0.0:1787826361.050174:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35098:x1874663346330496:12345-0@lo:101:mdt00_013.0 Request processed in 4962us (5014us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826361.050182:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24108 00000100:00100000:1.0:1787826361.050293:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850772680 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346330496:0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826361.050324:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.050334:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826361.054730:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332681472 00000100:00100000:3.0:1787826361.054744:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332681472 00010000:00010000:3.0:1787826361.054814:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 7 pid: 6528 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826361.054874:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332681344 00000100:00100000:3.0:1787826361.054878:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332681344 00010000:00010000:3.0:1787826361.054895:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 8 pid: 6527 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826361.054917:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22060 00000100:00100000:3.0:1787826361.054927:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332681472:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826361.054941:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826361.054948:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 7 pid: 6528 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.054959:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 7 pid: 6528 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.054981:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c2f7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801afb expref: 7 pid: 6528 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.055000:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826361.055007:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.055017:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826361.055085:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332681472:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 160us (5681us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.055093:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22060 00000100:00100000:3.0:1787826361.055144:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22061 00000100:00100000:3.0:1787826361.055152:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332681344:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826361.055159:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826361.055165:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 8 pid: 6527 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.055177:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 8 pid: 6527 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.055189:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c31a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b09 expref: 8 pid: 6527 timeout: 8273 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.055199:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826361.055204:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.055214:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.055694:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.055735:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826361.055751:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346330880:0@lo:105: 00000100:00100000:3.0:1787826361.055807:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826361.056008:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826361.056036:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346330880 00000100:00100000:3.0:1787826361.056043:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346330880 00000100:00100000:3.0:1787826361.056080:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7922 00000100:00100000:3.0:1787826361.056088:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346330880:12345-0@lo:105: 00010000:00010000:3.0:1787826361.056100:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.056139:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.056148:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.056156:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.056163:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.056171:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.056192:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490eea00) 00000100:00100000:3.0:1787826361.056494:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346330880:12345-0@lo:105: Request processed in 405us (685us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.056501:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7922 00000100:00100000:3.0:1787826361.056544:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346330880:0@lo:105: 00010000:00010000:3.0:1787826361.056559:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826361.056834:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332681344:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1671us (7372us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.056843:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22061 00000100:00080000:0.0:1787826361.057481:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787826361.058197:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.058211:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826361.058252:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787826361.058314:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.058322:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.058333:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826361.058356:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331008:0@lo:101:mdt00_013.0 00000100:00100000:0.0:1787826361.059717:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826361.059799:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826361.060034:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346331008 00000100:00100000:1.0:1787826361.060043:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346331008 00000100:00100000:1.0:1787826361.060299:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24109 00000100:00100000:1.0:1787826361.060321:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35098:x1874663346331008:12345-0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826361.060339:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826361.060362:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c336 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c32f expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.060389:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c336 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c32f expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.060403:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c336 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c32f expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.060412:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1b00) 00010000:00010000:1.0:1787826361.060415:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826361.060418:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826361.060579:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331008:12345-0@lo:101:mdt00_013.0 Request processed in 254us (859us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.060586:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24109 00000100:00100000:0.0:1787826361.061275:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331008:0@lo:101:mdt00_013.0 00010000:00010000:0.0:1787826361.061299:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.061325:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826361.061347:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826361.061396:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331136:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826361.061446:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826361.061923:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826361.061991:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346331136 00000100:00100000:1.0:1787826361.061998:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346331136 00000100:00100000:1.0:1787826361.062083:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24110 00000100:00100000:1.0:1787826361.062092:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331136:12345-0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826361.064071:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331136:12345-0@lo:1000:mdt00_013.0 Request processed in 1972us (2615us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.064084:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24110 00000100:00100000:3.0:1787826361.065843:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331136:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.065937:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331264:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.066006:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826361.066036:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826361.067371:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346331264 00000100:00100000:1.0:1787826361.067382:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346331264 00000100:00100000:1.0:1787826361.067504:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24111 00000100:00100000:1.0:1787826361.067516:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331264:12345-0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826361.067785:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331264:12345-0@lo:1000:mdt00_013.0 Request processed in 271us (1778us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826361.067793:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24111 00000100:00100000:3.0:1787826361.067863:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331264:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.067966:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331392:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.067998:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826361.070383:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.070491:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346331392 00000100:00100000:2.0:1787826361.070501:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346331392 00000100:00100000:2.0:1787826361.070554:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24112 00000100:00100000:2.0:1787826361.070564:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331392:12345-0@lo:1000:mdt00_013.0 00000100:00100000:2.0:1787826361.071865:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331392:12345-0@lo:1000:mdt00_013.0 Request processed in 1296us (3860us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826361.071876:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24112 00000100:00100000:3.0:1787826361.072032:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331392:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.072148:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331520:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.072187:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826361.072205:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.072358:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346331520 00000100:00100000:2.0:1787826361.072365:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346331520 00000100:00100000:2.0:1787826361.072405:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24113 00000100:00100000:2.0:1787826361.072414:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf89c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331520:12345-0@lo:1000:mdt00_013.0 00000100:00100000:2.0:1787826361.072796:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf89c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331520:12345-0@lo:1000:mdt00_013.0 Request processed in 382us (607us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.072803:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24113 00000100:00100000:3.0:1787826361.073315:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331520:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.073373:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331648:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.073402:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826361.073464:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.073499:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346331648 00000100:00100000:2.0:1787826361.073504:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346331648 00000100:00100000:2.0:1787826361.073564:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24114 00000100:00100000:2.0:1787826361.073636:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331648:12345-0@lo:1000:mdt00_013.0 00000100:00100000:2.0:1787826361.073783:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331648:12345-0@lo:1000:mdt00_013.0 Request processed in 211us (379us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.073791:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24114 00000100:00100000:3.0:1787826361.073861:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331648:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.073903:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331776:0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826361.073927:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826361.076001:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826361.078967:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346331776 00000100:00100000:0.0:1787826361.078983:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346331776 00000100:00100000:1.0:1787826361.079096:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24115 00000100:00100000:1.0:1787826361.079123:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331776:12345-0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826361.079273:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346331776:12345-0@lo:1000:mdt00_013.0 Request processed in 167us (5345us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.079282:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24115 00000100:00100000:3.0:1787826361.079456:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346331776:0@lo:1000:mdt00_013.0 00000040:00080000:3.0:1787826361.080333:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1231, 2168 off1124832 00000040:00080000:3.0:1787826361.080409:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1549, 2368 off1569048 00000004:00080000:3.0:1787826361.080415:0:35098:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784a358480 with version 6888 00000004:00080000:1.0:1787826361.080678:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6888 rpc_version 6888 00000020:00100000:1.0:1787826361.080718:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826361.080729:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7848cfea00 x1874663346332032/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826361.080753:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346332032:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826361.080817:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.080855:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826361.083294:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346332032 00000100:00100000:3.0:1787826361.083320:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346332032 00000100:00100000:3.0:1787826361.083397:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24116 00000100:00100000:3.0:1787826361.083409:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346332032:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826361.084420:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346332032:12345-0@lo:1000:osp_up0-1.0 Request processed in 1011us (3599us total) trans 8589957480 rc 0/0 00000100:00100000:3.0:1787826361.084429:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24116 00000100:00100000:1.0:1787826361.085278:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfea00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346332032:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787826361.085547:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.085563:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.087298:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.087350:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.087359:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.087366:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.087401:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.087407:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.087421:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826361.087446:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346332160:0@lo:101:mdt00_013.0 00000100:00100000:3.0:1787826361.087512:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826361.087548:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826361.087913:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346332160 00000100:00100000:1.0:1787826361.087921:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346332160 00000100:00100000:1.0:1787826361.087981:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24117 00000100:00100000:1.0:1787826361.087991:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35098:x1874663346332160:12345-0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826361.088012:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826361.088029:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c352 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c34b expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.088050:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c34b expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.088063:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c34b expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.088072:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f7845cb1400) 00010000:00010000:1.0:1787826361.088075:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826361.088079:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:1.0:1787826361.088137:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35098:x1874663346332160:12345-0@lo:101:mdt00_013.0 Request processed in 148us (624us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.088143:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24117 00000100:00100000:3.0:1787826361.088422:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd500 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346332160:0@lo:101:mdt00_013.0 00010000:00010000:3.0:1787826361.088438:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088447:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826361.088472:0:35098:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002a433d18 reply state 00000000f95b73e6 transno 4294985481 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088484:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088489:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:3.0:1787826361.088492:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826361.088495:0:35098:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826361.088501:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088510:0:35098:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826361.088526:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19a:0x0] 00000004:00010000:3.0:1787826361.088535:0:35098:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002a433d18 reply state 00000000f95b73e6 transno 4294985481 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088543:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088548:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:3.0:1787826361.088550:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826361.088552:0:35098:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787826361.088556:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5c:0x0]: returning size 4096 00000004:00080000:3.0:1787826361.088566:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826361.088586:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088593:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088598:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088621:0:35098:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826361.088626:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826361.088631:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.088638:0:35098:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.088721:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826361.088729:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.088737:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826361.088742:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826361.088750:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.088777:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.088784:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826361.088811:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.088819:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490eea00/0xe21309aa81f4c321 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c328 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.088832:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826361.088834:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826361.088835:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826361.088842:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826361.088854:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346332288:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826361.088896:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826361.088932:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346332288 00000100:00100000:2.0:1787826361.088937:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346332288 00000100:00100000:2.0:1787826361.088972:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11209 00000100:00100000:2.0:1787826361.088979:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c82c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346332288:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826361.088992:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826361.089007:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c328 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c321 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.089018:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826361.089021:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826361.089048:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c82c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346332288:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (151us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.089054:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11209 00000100:00100000:2.0:1787826361.089138:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e761f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346332288:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826361.089516:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226485:x1874663332681216:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 46055us (50252us total) trans 4294985481 rc 0/0 00000100:00100000:3.0:1787826361.089524:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68669 00000100:00100000:2.0:1787826361.094787:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826361.094893:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332681600 00000100:00100000:1.0:1787826361.094904:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332681600 00000100:00100000:1.0:1787826361.094968:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68670 00000100:00100000:1.0:1787826361.094980:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226474:x1874663332681600:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826361.095000:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826361.095023:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c359 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b10 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.095089:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c360 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.095103:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c360 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.095149:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c360 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826361.095199:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826361.095203:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826361.095460:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c360 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.095490:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c359 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b10 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.095506:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c360 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b10 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.095514:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826361.095517:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826361.095520:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:1.0:1787826361.095616:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226474:x1874663332681600:12345-192.168.203.56@tcp:101:bash.0 Request processed in 638us (839us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.095623:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68670 00000100:00100000:2.0:1787826361.106007:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826361.106135:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332681728 00000100:00100000:1.0:1787826361.106145:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332681728 00000100:00100000:1.0:1787826361.106212:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68671 00000100:00100000:1.0:1787826361.106234:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226474:x1874663332681728:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826361.106253:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826361.106276:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c367 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b17 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.106352:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.106367:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.106377:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.106389:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.106399:0:35080:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c336 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c32f expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.106408:0:35080:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.106441:0:35080:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c336 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4c32f expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826361.106458:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfc380 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000_UUID:35080:1874663346332416:0@lo:104: 00000100:00100000:1.0:1787826361.106518:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.106563:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.106840:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787826361.106854:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346332416 00000100:00100000:2.0:1787826361.106858:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346332416 00000100:00100000:2.0:1787826361.106895:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7923 00000100:00100000:2.0:1787826361.106902:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5c3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35080:x1874663346332416:12345-0@lo:104: 00010000:00010000:2.0:1787826361.106915:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.106970:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826361.106977:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5c3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:35080:x1874663346332416:12345-0@lo:104: Request processed in 76us (459us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.106983:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7923 00000100:00100000:1.0:1787826361.107247:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfc380 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000_UUID:35080:1874663346332416:0@lo:104: 00010000:00010000:1.0:1787826361.107266:0:35080:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.108241:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826361.108251:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.108260:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826361.108266:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826361.108275:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.108323:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.108330:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826361.108364:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.108371:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42600/0xe21309aa81f4c32f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c336 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.108395:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826361.108397:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826361.108398:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826361.108502:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346332544:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826361.108557:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826361.108689:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346332544 00000100:00100000:3.0:1787826361.108696:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346332544 00010000:00010000:3.0:1787826361.108726:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c336 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4c32f expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826361.108761:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22062 00000100:00100000:3.0:1787826361.108770:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3653:x1874663346332544:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826361.108782:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1787826361.111596:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957480 is committed 00010000:00010000:3.0:1787826361.111799:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c336 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4c32f expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.111832:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1b00) 00010000:00010000:3.0:1787826361.111838:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.111848:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.111855:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.111895:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826361.111980:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346332544:12345-0@lo:103:ldlm_bl.0 Request processed in 3209us (3420us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.111994:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22062 00010000:00010000:0.0:1787826361.112003:0:35080:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.112017:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826361.112245:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346332544:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826361.112385:0:35080:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-mdtlov_UUID:35080:1874663346332672:0@lo:1000:mdt00_008.0 00000100:00100000:0.0:1787826361.112458:0:35080:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826361.112497:0:35080:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.112563:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346332672 00000100:00100000:2.0:1787826361.112572:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346332672 00000100:00100000:2.0:1787826361.112620:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24118 00000100:00100000:2.0:1787826361.112630:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:35080:x1874663346332672:12345-0@lo:1000:mdt00_008.0 00000100:00100000:2.0:1787826361.112900:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:35080:x1874663346332672:12345-0@lo:1000:mdt00_008.0 Request processed in 271us (442us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.112907:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24118 00000100:00100000:0.0:1787826361.113730:0:35080:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-mdtlov_UUID:35080:1874663346332672:0@lo:1000:mdt00_008.0 00010000:00010000:0.0:1787826361.113808:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c37c lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.113827:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c37c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.113840:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c37c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826361.113870:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c37c lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.113886:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.113898:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.113905:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.113984:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.113992:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826361.113997:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114005:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114020:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826361.114095:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826361.114101:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114282:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c36e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114300:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114318:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114325:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114332:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826361.114335:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114342:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114352:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7845cb1b00) 00010000:00010000:0.0:1787826361.114355:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826361.114358:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114364:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c375 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114412:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c367 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b17 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114428:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c37c lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b17 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.114437:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:0.0:1787826361.114440:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826361.114442:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826361.114696:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226474:x1874663332681728:12345-192.168.203.56@tcp:101:bash.0 Request processed in 8473us (8697us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826361.114707:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68671 00000100:00100000:2.0:1787826361.135393:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826361.135483:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332681856 00000100:00100000:0.0:1787826361.135495:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332681856 00000100:00100000:0.0:1787826361.135566:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68672 00000100:00100000:0.0:1787826361.135587:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226474:x1874663332681856:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826361.135617:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826361.135642:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c383 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b1e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.135717:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c38a lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.135732:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c38a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.135746:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c38a lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826361.135871:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5c:0x0]: returning size 4096 00000004:00200000:0.0:1787826361.135877:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826361.135922:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c383 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b1e expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.135945:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c38a lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b1e expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.135954:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:0.0:1787826361.135957:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826361.135961:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826361.136048:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226474:x1874663332681856:12345-192.168.203.56@tcp:101:bash.0 Request processed in 463us (668us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826361.136059:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68672 00000100:00100000:3.0:1787826361.141255:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826361.141422:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332681984 00000100:00100000:1.0:1787826361.141435:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332681984 00000100:00100000:1.0:1787826361.141515:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68673 00000100:00100000:1.0:1787826361.141529:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226474:x1874663332681984:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826361.141568:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826361.141601:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c391 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b25 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.141673:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c398 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.141689:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c398 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.141702:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c398 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826361.141766:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19c:0x0]: returning size 4096 00000004:00200000:1.0:1787826361.141772:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826361.141814:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c391 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b25 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.141835:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c398 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b25 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.141845:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:1.0:1787826361.141848:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826361.141852:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826361.142570:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226474:x1874663332681984:12345-192.168.203.56@tcp:101:bash.0 Request processed in 1039us (1323us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.142635:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68673 00000100:00100000:1.0:1787826361.144705:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826361.144811:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332682112 00000100:00100000:3.0:1787826361.144834:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332682112 00000100:00100000:3.0:1787826361.144900:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68674 00000100:00100000:3.0:1787826361.144912:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226474:x1874663332682112:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826361.144933:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826361.144955:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c39f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b2c expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.145019:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.145033:0:8752:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c352 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c34b expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.145184:0:8752:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.145222:0:8752:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4c34b expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826361.145239:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346332800:0@lo:104: 00000100:00100000:3.0:1787826361.145316:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.145380:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346332800 00000100:00100000:3.0:1787826361.145385:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826361.145390:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346332800 00000100:00100000:1.0:1787826361.145516:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7924 00000100:00100000:1.0:1787826361.145527:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346332800:12345-0@lo:104: 00010000:00010000:1.0:1787826361.145548:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826361.145655:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826361.145662:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:8752:x1874663346332800:12345-0@lo:104: Request processed in 137us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.145669:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7924 00000100:00100000:3.0:1787826361.146597:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dd880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8752:1874663346332800:0@lo:104: 00010000:00010000:3.0:1787826361.146624:0:8752:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.147715:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826361.147729:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.147741:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826361.147750:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826361.147760:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.147811:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.147818:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826361.147858:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 1/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.147867:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c34b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c352 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.147892:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826361.147894:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826361.147895:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826361.147906:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787826361.147927:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346332928:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826361.147997:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826361.148053:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346332928 00000100:00100000:0.0:1787826361.148061:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346332928 00010000:00010000:0.0:1787826361.148093:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c352 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4c34b expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826361.148173:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22063 00000100:00100000:0.0:1787826361.148183:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346332928:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826361.148196:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826361.155480:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c352 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4c34b expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.155517:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f7845cb1400) 00010000:00010000:0.0:1787826361.155523:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.155535:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.155543:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.155593:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826361.155700:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346332928:12345-0@lo:103:ldlm_bl.0 Request processed in 7516us (7701us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826361.155710:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22063 00010000:00010000:3.0:1787826361.155796:0:8752:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.155806:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826361.155835:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346332928:0@lo:103:ldlm_bl.0 00000004:00200000:3.0:1787826361.155873:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x19a:0x0]: returning size 4096 00000004:00200000:3.0:1787826361.155880:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826361.155944:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c39f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b2c expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.155963:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b2c expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.155972:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f7845cb1400) 00010000:00010000:3.0:1787826361.155975:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826361.155978:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826361.156043:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226474:x1874663332682112:12345-192.168.203.56@tcp:101:bash.0 Request processed in 11134us (11350us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.156051:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68674 00000100:00100000:2.0:1787826361.165482:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826361.165626:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332682368 00000100:00100000:0.0:1787826361.165638:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332682368 00000100:00100000:0.0:1787826361.165707:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68675 00000100:00100000:0.0:1787826361.165720:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226474:x1874663332682368:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826361.165744:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826361.165748:0:6527:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826361.165790:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68000/0xe21309aa81f4c3a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b2c expref: 9 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.165812:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f7845cb1400) 00010000:00010000:0.0:1787826361.165823:0:6527:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826361.165840:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c3ad lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b33 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.166004:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.166021:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.166031:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.166045:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.166053:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.166060:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826361.166530:0:6527:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x523:0x0], index 0: 0 00010000:00010000:0.0:1787826361.168511:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c3c2 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.168535:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c3c2 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.168549:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c3c2 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.168618:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.168626:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.168736:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169360:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169368:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826361.169373:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169382:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169397:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f794b57a400) 00010000:00010000:0.0:1787826361.169400:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826361.169404:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169412:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3b4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826361.169430:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e044ca3d reply state 0000000067c7a3fb transno 8589957481 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169442:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169450:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f794b57ac00) 00010000:00010000:0.0:1787826361.169452:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826361.169455:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826361.169506:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c3ad lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b33 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169530:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c3c2 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b33 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.169538:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:0.0:1787826361.169540:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826361.169543:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:0.0:1787826361.169794:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226474:x1874663332682368:12345-192.168.203.56@tcp:101:bash.0 Request processed in 4076us (4326us total) trans 8589957481 rc 0/0 00000100:00100000:0.0:1787826361.169812:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68675 00000100:00100000:0.0:1787826361.174924:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826361.175047:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332682496 00000100:00100000:2.0:1787826361.175058:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332682496 00000100:00100000:2.0:1787826361.175159:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30884 00000100:00100000:2.0:1787826361.175170:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226474:x1874663332682496:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:2.0:1787826361.175188:0:8444:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826361.175211:0:8444:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b3a expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787826361.175237:0:8444:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x523:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787826361.175258:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=8173s v=(0 0) 00010000:00010000:2.0:1787826361.175263:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787826361.175273:0:8444:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 7 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.175310:0:8444:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 7 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826361.175325:0:8444:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826361.175390:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226474:x1874663332682496:12345-192.168.203.56@tcp:101:bash.0 Request processed in 223us (475us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.175398:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30884 00000100:00100000:1.0:1787826361.188876:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826361.189073:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332682752 00000100:00100000:0.0:1787826361.189083:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332682752 00000100:00100000:0.0:1787826361.189180:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9394 00000100:00100000:0.0:1787826361.189191:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226474:x1874663332682752:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:0.0:1787826361.189507:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226474:x1874663332682752:12345-192.168.203.56@tcp:35:bash.0 Request processed in 317us (640us total) trans 8589957482 rc 0/0 00000100:00100000:0.0:1787826361.189516:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9394 00000100:00100000:2.0:1787826361.195251:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826361.195600:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332683008 00000100:00100000:0.0:1787826361.195611:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332683008 00000100:00100000:0.0:1787826361.195684:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68676 00000100:00100000:0.0:1787826361.195694:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226474:x1874663332683008:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826361.195714:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826361.195717:0:35092:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826361.195748:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c398 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b25 expref: 10 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.195768:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:0.0:1787826361.195773:0:35092:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826361.195788:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c3d7 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b41 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.195930:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.195942:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.195951:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.195964:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.195972:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.195978:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826361.196161:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346333056:0@lo:1000:mdt00_009.0 00000100:00100000:0.0:1787826361.196221:0:35092:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826361.196258:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78434e4cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826361.196933:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346333056 00000100:00100000:2.0:1787826361.196941:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346333056 00000100:00100000:2.0:1787826361.196995:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24119 00000100:00100000:2.0:1787826361.197004:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346333056:12345-0@lo:1000:mdt00_009.0 00000100:00100000:2.0:1787826361.197091:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35092:x1874663346333056:12345-0@lo:1000:mdt00_009.0 Request processed in 88us (869us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.197099:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24119 00000100:00100000:0.0:1787826361.199157:0:35092:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957480 gen 2 00000004:00080000:0.0:1787826361.199170:0:35092:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957480 committed transno 8589957480 00000100:00100000:0.0:1787826361.199212:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-mdtlov_UUID:35092:1874663346333056:0@lo:1000:mdt00_009.0 00000004:00080000:0.0:1787826361.199527:0:35092:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x4ce:0x0], index 1: 0 00010000:00010000:0.0:1787826361.199722:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c3ec lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199737:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c3ec lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199748:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c3ec lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199786:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199793:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199800:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199807:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199812:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826361.199818:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199824:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199836:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:0.0:1787826361.199839:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826361.199843:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199850:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42600/0xe21309aa81f4c3de lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826361.199864:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000007012922 reply state 0000000070f74ec0 transno 4294985482 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199875:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199881:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x4ec94fd6 (ffff9f794b57a400) 00010000:00010000:0.0:1787826361.199883:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826361.199886:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826361.199922:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c3d7 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b41 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199936:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c3ec lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b41 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826361.199944:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:0.0:1787826361.199947:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826361.199950:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42800, rc 0) 00000100:00100000:0.0:1787826361.200014:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226474:x1874663332683008:12345-192.168.203.56@tcp:101:bash.0 Request processed in 4321us (4772us total) trans 4294985482 rc 0/0 00000100:00100000:0.0:1787826361.200023:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68676 00000100:00100000:1.0:1787826361.217594:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826361.218069:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332683136 00000100:00100000:3.0:1787826361.218080:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332683136 00000100:00100000:3.0:1787826361.218181:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30885 00000100:00100000:3.0:1787826361.218194:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:226474:x1874663332683136:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826361.218211:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826361.218234:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b48 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826361.218266:0:8445:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4ce:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:3.0:1787826361.218289:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ce:0x0].0x0 since=8173s v=(0 0) 00010000:00010000:3.0:1787826361.218294:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ce:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826361.218313:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.218330:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.218343:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826361.218397:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226474:x1874663332683136:12345-192.168.203.56@tcp:101:bash.0 Request processed in 207us (814us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.218403:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30885 00000100:00080000:0.0:1787826361.218470:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826361.228762:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826361.228860:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332683392 00000100:00100000:3.0:1787826361.228871:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332683392 00000100:00100000:3.0:1787826361.228933:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9395 00000100:00100000:3.0:1787826361.228944:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226474:x1874663332683392:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:3.0:1787826361.229298:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:226474:x1874663332683392:12345-192.168.203.56@tcp:35:bash.0 Request processed in 354us (545us total) trans 4294985483 rc 0/0 00000100:00100000:3.0:1787826361.229314:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9395 00000100:00100000:1.0:1787826361.281967:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826361.282015:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826361.282064:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826361.282605:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332683520 00000100:00100000:3.0:1787826361.282616:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332683520 00000100:00100000:3.0:1787826361.282698:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332683648 00000100:00100000:3.0:1787826361.282702:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332683648 00000100:00100000:3.0:1787826361.282729:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332683776 00000100:00100000:3.0:1787826361.282732:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332683776 00000100:00100000:3.0:1787826361.282758:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11210 00000100:00100000:3.0:1787826361.282768:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:6166:x1874663332683520:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826361.282783:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826361.282809:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c360 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b10 expref: 12 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.282832:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826361.282844:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c3c2 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b33 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.282855:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f794b57a800) 00010000:00010000:3.0:1787826361.282871:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c37c lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b17 expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.282878:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a900) 00010000:00010000:3.0:1787826361.282884:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826361.282943:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:6166:x1874663332683520:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 177us (985us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.282950:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11210 00000100:00100000:3.0:1787826361.282969:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11211 00000100:00100000:3.0:1787826361.282976:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:15673:x1874663332683648:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826361.282983:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826361.282995:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c1d8 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dc1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ac3 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.283011:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc1:0x0].0x0 (ffff9f7952ca9700) 00010000:00010000:3.0:1787826361.283019:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826361.283036:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:15673:x1874663332683648:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 61us (1021us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.283054:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11211 00000100:00100000:3.0:1787826361.283069:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11212 00000100:00100000:3.0:1787826361.283076:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:6167:x1874663332683776:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826361.283084:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826361.283096:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c3ec lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b41 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.283145:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f794b57ad00) 00010000:00010000:3.0:1787826361.283159:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c38a lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b1e expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.283170:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:3.0:1787826361.283179:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c145 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5a:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801aa0 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826361.283188:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5a:0x0].0x0 (ffff9f7967ac8600) 00010000:00010000:3.0:1787826361.283194:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826361.283213:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:6167:x1874663332683776:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 138us (1148us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826361.283219:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11212 00000100:00100000:1.0:1787826361.325309:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346333312:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.325401:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.325444:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850773100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346333824:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.325469:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.325489:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346334336:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.325514:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.325539:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1787826361.325545:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850770700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346333440:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.325566:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.325595:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346333952:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.325615:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.325630:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346334464:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.325663:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826361.325694:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346333312 00000100:00100000:1.0:1787826361.325700:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346333312 00000100:00100000:1.0:1787826361.325742:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346333824 00000100:00100000:1.0:1787826361.325744:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346333824 00000100:00100000:1.0:1787826361.325770:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346333440 00000100:00100000:1.0:1787826361.325773:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346333440 00000100:00100000:1.0:1787826361.325791:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346333952 00000100:00100000:1.0:1787826361.325793:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346333952 00000100:00100000:1.0:1787826361.325811:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346334464 00000100:00100000:1.0:1787826361.325814:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346334464 00000100:00100000:1.0:1787826361.325847:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14378 00000100:00100000:1.0:1787826361.325856:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346333312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.325907:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346333312:12345-0@lo:400:kworker.0 Request processed in 52us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.325913:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14378 00000100:00100000:1.0:1787826361.325929:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14379 00000100:00100000:1.0:1787826361.325936:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346333824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.325963:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346333824:12345-0@lo:400:kworker.0 Request processed in 28us (494us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.325971:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14379 00000100:00100000:1.0:1787826361.325983:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14380 00000100:00100000:1.0:1787826361.325989:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346333440:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.326011:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346333440:12345-0@lo:400:kworker.0 Request processed in 23us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.326016:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14380 00000100:00100000:1.0:1787826361.326027:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14381 00000100:00100000:1.0:1787826361.326032:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346333952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.326057:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346333952:12345-0@lo:400:kworker.0 Request processed in 25us (441us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.326062:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14381 00000100:00100000:1.0:1787826361.326074:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14382 00000100:00100000:1.0:1787826361.326080:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346334464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.326138:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346334464:12345-0@lo:400:kworker.0 Request processed in 25us (441us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826361.326144:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14382 00000100:00100000:1.0:1787826361.326185:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850770700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346333440:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.326195:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346333952:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.326205:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346334464:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.326214:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346333312:0@lo:400:kworker.0 00000100:00100000:1.0:1787826361.326221:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850773100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346333824:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.326524:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346334336 00000100:00100000:2.0:1787826361.326529:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346334336 00000100:00100000:2.0:1787826361.326579:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8882 00000100:00100000:2.0:1787826361.326587:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346334336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.326646:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346334336:12345-0@lo:400:kworker.0 Request processed in 49us (1119us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.326653:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8882 00000100:00100000:2.0:1787826361.326682:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346333184:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.326714:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826361.326744:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346333696:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.326767:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826361.326786:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346334208:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.326807:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826361.326839:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1787826361.326845:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850772680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346333568:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.326867:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826361.326882:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346334080:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.326900:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826361.326924:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346333184 00000100:00100000:2.0:1787826361.326928:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346333184 00000100:00100000:2.0:1787826361.326956:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2241 00000100:00100000:2.0:1787826361.326962:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784331f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346333184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.326997:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784331f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346333184:12345-0@lo:400:kworker.0 Request processed in 36us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.327004:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2241 00000100:00100000:2.0:1787826361.327031:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346333696 00000100:00100000:2.0:1787826361.327034:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346333696 00000100:00100000:2.0:1787826361.327060:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8883 00000100:00100000:2.0:1787826361.327066:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346333696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.327094:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346333696:12345-0@lo:400:kworker.0 Request processed in 29us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.327100:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8883 00000100:00100000:2.0:1787826361.327135:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346334208 00000100:00100000:2.0:1787826361.327139:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346334208 00000100:00100000:2.0:1787826361.327157:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346334080 00000100:00100000:2.0:1787826361.327159:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346334080 00000100:00100000:2.0:1787826361.327176:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8884 00000100:00100000:2.0:1787826361.327182:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346334208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.327208:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346334208:12345-0@lo:400:kworker.0 Request processed in 26us (400us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.327213:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8884 00000100:00100000:2.0:1787826361.327223:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8885 00000100:00100000:2.0:1787826361.327229:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346334080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.327252:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346334080:12345-0@lo:400:kworker.0 Request processed in 24us (352us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826361.327258:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8885 00000100:00100000:2.0:1787826361.327294:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346334336:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.327332:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346334080:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.327340:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346333184:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.327348:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346333696:0@lo:400:kworker.0 00000100:00100000:2.0:1787826361.327356:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973860 gen 2 00000004:00080000:2.0:1787826361.327360:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f785035ad80, transno 4294973860 00000100:00100000:2.0:1787826361.327373:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346334208:0@lo:400:kworker.0 00000100:00080000:0.0:1787826361.327764:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787826361.327769:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787826361.327771:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787826361.327774:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826361.327784:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346333568 00000100:00100000:0.0:1787826361.327789:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346333568 00000100:00100000:0.0:1787826361.327846:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14383 00000100:00100000:0.0:1787826361.327853:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346333568:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826361.327899:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346333568:12345-0@lo:400:kworker.0 Request processed in 47us (1032us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826361.327906:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14383 00000100:00100000:0.0:1787826361.327945:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850772680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346333568:0@lo:400:kworker.0 00000100:00080000:2.0:1787826361.328052:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1787826361.520998:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826361.838473:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346334592:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826361.838684:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826361.838944:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346334592 00000100:00100000:0.0:1787826361.838952:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346334592 00000100:00100000:0.0:1787826361.839012:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6263 00000100:00100000:0.0:1787826361.839022:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346334592:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826361.839050:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826361.839318:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346334592:12345-0@lo:13:osp-pre-1-1.0 Request processed in 296us (749us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826361.839326:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6263 00000100:00100000:3.0:1787826361.839510:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346334592:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826362.354160:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346334720:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826362.354237:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826362.354307:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346334848:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826362.354333:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826362.354354:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826362.354360:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785035ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346334976:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826362.354385:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826362.354422:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346334720 00000100:00100000:2.0:1787826362.354428:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346334720 00000100:00100000:2.0:1787826362.354479:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346334848 00000100:00100000:2.0:1787826362.354482:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346334848 00000100:00100000:2.0:1787826362.354502:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24120 00000100:00100000:2.0:1787826362.354511:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346334720:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826362.354543:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443364864 avail 1297125376 00000100:00100000:2.0:1787826362.354588:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346334720:12345-0@lo:41:osp-pre-1-0.0 Request processed in 77us (350us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826362.354594:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24120 00000100:00100000:2.0:1787826362.354611:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24121 00000100:00100000:2.0:1787826362.354618:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346334848:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826362.354635:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:2.0:1787826362.354666:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346334848:12345-0@lo:41:osp-pre-0-1.0 Request processed in 48us (332us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826362.354672:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24121 00000100:00100000:2.0:1787826362.354692:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346334976 00000100:00100000:2.0:1787826362.354696:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346334976 00000100:00100000:2.0:1787826362.354720:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6264 00000100:00100000:2.0:1787826362.354725:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346334976:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826362.354742:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3893977088 avail 3662454784 00000100:00100000:2.0:1787826362.354771:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346334976:12345-0@lo:13:osp-pre-0-0.0 Request processed in 45us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826362.354778:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6264 00000100:00100000:2.0:1787826362.354824:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785035ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346334976:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826362.354836:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346334848:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826362.354869:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346334720:0@lo:41:osp-pre-1-0.0 00000004:00080000:3.0:1787826362.355334:0:8751:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8174567227636ns, next at 8175567224891ns, rc = 0 00000100:00100000:3.0:1787826362.355374:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346335104:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826362.355409:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826362.355433:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826362.355438:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346335232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826362.355461:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826362.355491:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346335104 00000100:00100000:3.0:1787826362.355495:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346335104 00000100:00100000:3.0:1787826362.355519:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346335232 00000100:00100000:3.0:1787826362.355521:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346335232 00000100:00100000:3.0:1787826362.355536:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6265 00000100:00100000:3.0:1787826362.355542:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346335104:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826362.355553:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484846144 pending 0 free 3893977088 avail 3662454784 00000100:00100000:3.0:1787826362.355585:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346335104:12345-0@lo:13:osp-pre-0-1.0 Request processed in 43us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826362.355591:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6265 00000100:00100000:3.0:1787826362.355605:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6266 00000100:00100000:3.0:1787826362.355611:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346335232:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787826362.355621:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826362.355645:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346335232:12345-0@lo:13:osp-pre-1-0.0 Request processed in 35us (183us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826362.355650:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6266 00000100:00100000:3.0:1787826362.355684:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ded80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346335232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826362.355696:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346335104:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1787826362.547885:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826362.840252:0:167417:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:2.0:1787826362.840302:0:167417:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:2.0:1787826363.566275:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826364.588388:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1787826364.910742:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 370 committed 00010000:00010000:1.0:1787826364.910801:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c287 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826364.910818:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c287 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826364.911079:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346335360:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1787826364.911183:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826364.911267:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346335360 00000100:00100000:0.0:1787826364.911285:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346335360 00000100:00100000:0.0:1787826364.911342:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30886 00000100:00100000:0.0:1787826364.911354:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346335360:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1787826364.911379:0:8445:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1232 count 1 00010000:00010000:0.0:1787826364.911414:0:8445:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826364.911436:0:8445:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826364.911445:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4d0:0x0].0x0 since=11s v=(0 0) 00010000:00010000:0.0:1787826364.911450:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826364.911454:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826364.911483:0:8445:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 6 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826364.911497:0:8445:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 6 pid: 24218 timeout: 8277 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826364.911509:0:8445:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346335488:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826364.911583:0:8445:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7540 going to sleep for 11 seconds 00000100:00080000:0.0:1787826364.911598:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787826364.911778:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911787:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911794:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911798:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826364.911803:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911810:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911825:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f79785ccf00) 00010000:00010000:3.0:1787826364.911829:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826364.911833:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911838:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c33d lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911871:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911878:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911888:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911893:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826364.911896:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911902:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911909:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:3.0:1787826364.911911:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826364.911914:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911919:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c344 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911934:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911940:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911945:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911950:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826364.911952:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911961:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911968:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x4ec94fd6 (ffff9f794b57a400) 00010000:00010000:3.0:1787826364.911970:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826364.911972:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.911977:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c3e5 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.912177:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c287 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.912187:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826364.912207:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e69000/0xe21309aa81f4c280 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c287 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826364.912222:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985483 is committed 00000020:00080000:1.0:1787826364.912244:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784a358780 refcount 2 committed 0 result 0 batchid 3445 00000020:00080000:1.0:1787826364.912247:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a358480 00000020:00080000:1.0:1787826364.912251:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1549 00000020:00080000:1.0:1787826364.912253:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78475dbe00 00000020:00080000:1.0:1787826364.912256:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1231 00000001:00080000:1.0:1787826364.912267:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985480 is committed 00000020:00080000:1.0:1787826364.912272:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f784d067180 refcount 2 committed 0 result 0 batchid 3444 00000020:00080000:1.0:1787826364.912287:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d067a80 00000020:00080000:1.0:1787826364.912290:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1548 00000020:00080000:1.0:1787826364.912293:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff200 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7855a6a000 00000020:00080000:1.0:1787826364.912295:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1230 00000004:00080000:1.0:1787826364.912304:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 369 committed 00000020:00080000:1.0:1787826364.912313:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78434e49c0 refcount 2 committed 0 result 0 batchid 3443 00000020:00080000:1.0:1787826364.912315:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed6a000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78434e4c00 00000020:00080000:1.0:1787826364.912318:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1547 00000020:00080000:1.0:1787826364.912321:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed6ae00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d006000 00000020:00080000:1.0:1787826364.912323:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1229 00000020:00080000:1.0:1787826364.912329:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78434e4a80 refcount 2 committed 0 result 0 batchid 3442 00000020:00080000:1.0:1787826364.912331:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed6a700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78434e4600 00000020:00080000:1.0:1787826364.912334:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1546 00000020:00080000:1.0:1787826364.912336:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796ed6ac00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d007000 00000020:00080000:1.0:1787826364.912338:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1228 00000001:00080000:1.0:1787826364.912346:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985474 is committed 00000020:00080000:1.0:1787826364.912352:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3441 batchid updated 00000100:00100000:3.0:1787826364.912847:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826364.912877:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346335616:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787826364.912938:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826364.912983:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346335616 00000100:00100000:3.0:1787826364.912990:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346335616 00000100:00100000:3.0:1787826364.913038:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11213 00000100:00100000:3.0:1787826364.913046:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3652:x1874663346335616:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787826364.913058:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826364.913079:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c287 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x199:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c280 expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826364.913093:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x199:0x0].0x5a734ce9 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826364.913101:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826364.913149:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346335616:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 104us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826364.913156:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11213 00000100:00100000:3.0:1787826364.913200:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346335616:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:1.0:1787826364.913763:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3445 committed batchid 3441 00000004:00080000:1.0:1787826364.915857:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846318240 with version 6889 00000020:00080000:1.0:1787826364.915885:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3439 cancel update log [0x200000401:0x3:0x0].1543: rc = 0 00000004:00080000:2.0:1787826364.916054:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6889 rpc_version 6889 00000020:00100000:2.0:1787826364.916089:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826364.916100:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784836f800 x1874663346335872/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000020:00080000:1.0:1787826364.916438:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3439 cancel update log [0x240000400:0x3:0x0].1225: rc = 0 00000004:00080000:1.0:1787826364.916471:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846318a80 with version 6890 00000020:00080000:1.0:1787826364.916474:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3440 cancel update log [0x200000401:0x3:0x0].1544: rc = 0 00000020:00080000:1.0:1787826364.916495:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3440 cancel update log [0x240000400:0x3:0x0].1226: rc = 0 00000004:00080000:1.0:1787826364.916515:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7846318840 with version 6891 00000020:00080000:1.0:1787826364.916518:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3441 cancel update log [0x200000401:0x3:0x0].1545: rc = 0 00000020:00080000:1.0:1787826364.916534:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3441 cancel update log [0x240000400:0x3:0x0].1227: rc = 0 00000020:00080000:1.0:1787826364.916543:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3441 00000100:00100000:2.0:1787826364.916908:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346335872:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826364.917364:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826364.917402:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851ff5000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826364.917716:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346335872 00000100:00100000:3.0:1787826364.917724:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346335872 00000100:00100000:3.0:1787826364.919751:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24122 00000100:00100000:3.0:1787826364.919771:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346335872:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826364.919982:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346335872:12345-0@lo:1000:osp_up0-1.0 Request processed in 217us (2617us total) trans 8589957483 rc 0/0 00000100:00100000:3.0:1787826364.919989:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24122 00000100:00100000:3.0:1787826364.920084:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346335872:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826364.920099:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6890 rpc_version 6890 00000020:00100000:3.0:1787826364.920140:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826364.920147:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b19f480 x1874663346336128/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826364.920656:0:8445:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8445:1874663346335488:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826364.920682:0:8445:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826364.920813:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346336128:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826364.920862:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826364.920895:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826364.921009:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346336128 00000100:00100000:2.0:1787826364.921016:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346336128 00000100:00100000:2.0:1787826364.921305:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24123 00000100:00100000:2.0:1787826364.921316:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346336128:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826364.921458:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346336128:12345-0@lo:1000:osp_up0-1.0 Request processed in 145us (595us total) trans 8589957484 rc 0/0 00000100:00100000:2.0:1787826364.921465:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24123 00000100:00100000:3.0:1787826364.921645:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346336128:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826364.921657:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6891 rpc_version 6891 00000020:00100000:3.0:1787826364.921673:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826364.921679:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b19c000 x1874663346336384/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826364.921693:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346336384:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826364.921826:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826364.921958:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851ff5000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826364.922190:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826364.922260:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346336384 00000100:00100000:1.0:1787826364.922271:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346336384 00000100:00100000:1.0:1787826364.922318:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24124 00000100:00100000:1.0:1787826364.922454:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346336384:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826364.922601:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346336384:12345-0@lo:1000:osp_up0-1.0 Request processed in 275us (876us total) trans 8589957485 rc 0/0 00000100:00100000:1.0:1787826364.922609:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24124 00000100:00100000:1.0:1787826364.922929:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332683904 00000100:00100000:1.0:1787826364.922935:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332683904 00010000:00010000:1.0:1787826364.922986:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 7 pid: 24218 timeout: 8277 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826364.923018:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22064 00000100:00100000:1.0:1787826364.923027:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c80050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332683904:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826364.923039:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826364.923045:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 7 pid: 24218 timeout: 8277 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923060:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 7 pid: 24218 timeout: 8277 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923088:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008837638e/0xe21309aa81f4c0a4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801a61 expref: 7 pid: 24218 timeout: 8277 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923325:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826364.923404:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d0:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826364.923411:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923421:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923450:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826364.923559:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c80050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332683904:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 532us (1369us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826364.923566:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22064 00010000:00010000:1.0:1787826364.923727:0:8445:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923737:0:8445:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923752:0:8445:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923760:0:8445:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923780:0:8445:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826364.923939:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346335360:12345-0@lo:6:osp-syn-0-1.0 Request processed in 12586us (12753us total) trans 4294973862 rc 0/0 00000100:00100000:1.0:1787826364.923947:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30886 00010000:00010000:1.0:1787826364.923970:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826364.923977:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923985:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826364.923990:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.923999:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.924011:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.924017:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0d22958/0xe21309aa81f4c401 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4d0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826364.924035:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826364.924036:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826364.924038:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826364.924125:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826364.924755:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346336384:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826364.924794:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784a11ea00/2, rc 0, transno 4294973862 00000100:00100000:3.0:1787826364.924800:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346335360:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1787826365.331383:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826365.332673:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332684032 00000100:00100000:3.0:1787826365.332687:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332684032 00000100:00100000:3.0:1787826365.332774:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68677 00000100:00100000:3.0:1787826365.332787:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226655:x1874663332684032:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826365.332818:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826365.332840:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c408 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b4f expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.332911:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.332925:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.332937:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826365.332987:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826365.332991:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826365.333012:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.333035:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c408 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.333051:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.333060:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826365.333062:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826365.333066:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68a00, rc 0) 00000100:00100000:3.0:1787826365.333149:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226655:x1874663332684032:12345-192.168.203.56@tcp:101:cat.0 Request processed in 365us (1778us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.333156:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68677 00000100:00080000:0.0:1787826365.333657:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826365.352322:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826365.352392:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332684160 00000100:00100000:0.0:1787826365.352402:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332684160 00000100:00100000:0.0:1787826365.352466:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68678 00000100:00100000:0.0:1787826365.352477:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226655:x1874663332684160:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826365.352497:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826365.352518:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c416 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b56 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352581:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352594:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352605:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352630:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352637:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352644:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352719:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c42b lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352728:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c42b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352739:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c42b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826365.352762:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c42b lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352777:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352783:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352790:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352807:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352814:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826365.352819:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352826:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352838:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826365.352841:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826365.352845:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352851:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c41d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352864:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352869:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352875:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352880:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826365.352882:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352888:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352895:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a100) 00010000:00010000:0.0:1787826365.352898:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826365.352901:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352906:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c424 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352936:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c416 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b56 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352949:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c42b lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b56 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.352962:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:0.0:1787826365.352965:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826365.352967:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00100000:0.0:1787826365.353025:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226655:x1874663332684160:12345-192.168.203.56@tcp:101:cat.0 Request processed in 550us (713us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826365.353032:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68678 00000100:00100000:2.0:1787826365.357092:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826365.357676:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332684288 00000100:00100000:1.0:1787826365.357686:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332684288 00000100:00080000:0.0:1787826365.357877:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826365.359049:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68679 00000100:00100000:0.0:1787826365.359064:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843435c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:226655:x1874663332684288:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826365.359090:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826365.359132:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4c432 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b5d expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.359199:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c439 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.359212:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c439 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.359223:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c439 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826365.359573:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5c:0x0]: returning size 4096 00000004:00200000:0.0:1787826365.359579:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826365.360346:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4c432 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b5d expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.360376:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c439 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b5d expref: 6 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.360385:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:0.0:1787826365.360388:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826365.360391:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43600, rc 0) 00000100:00100000:0.0:1787826365.360456:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843435c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226655:x1874663332684288:12345-192.168.203.56@tcp:101:cat.0 Request processed in 1397us (3372us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.373648:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826365.373688:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332684416 00000100:00100000:3.0:1787826365.373698:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332684416 00000100:00100000:3.0:1787826365.373761:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68680 00000100:00100000:3.0:1787826365.373772:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226655:x1874663332684416:12345-192.168.203.56@tcp:101:cat.0 00000100:00100000:0.0:1787826365.373779:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68679 00010000:00010000:3.0:1787826365.373793:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826365.373818:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c440 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b64 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.373881:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.373904:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.373916:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826365.375835:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19c:0x0]: returning size 4096 00000004:00200000:3.0:1787826365.375846:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826365.376031:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c440 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.376054:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.376064:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826365.376067:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826365.376070:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826365.376165:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226655:x1874663332684416:12345-192.168.203.56@tcp:101:cat.0 Request processed in 2393us (2527us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.376173:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68680 00000100:00100000:2.0:1787826365.380464:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826365.380681:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332684544 00000100:00100000:3.0:1787826365.380690:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332684544 00000100:00100000:3.0:1787826365.380765:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68681 00000100:00100000:3.0:1787826365.380776:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843435880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226655:x1874663332684544:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826365.380797:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826365.380825:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c44e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b6b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.381089:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.381129:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.381141:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826365.381195:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x19a:0x0]: returning size 4096 00000004:00200000:3.0:1787826365.381200:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826365.381240:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c44e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.381257:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.381266:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:3.0:1787826365.381268:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826365.381281:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68600, rc 0) 00000100:00100000:3.0:1787826365.381504:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843435880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226655:x1874663332684544:12345-192.168.203.56@tcp:101:cat.0 Request processed in 728us (1044us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.381511:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68681 00000100:00100000:0.0:1787826365.396164:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826365.396364:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332684800 00000100:00100000:1.0:1787826365.396383:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332684800 00000100:00100000:1.0:1787826365.396451:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68682 00000100:00100000:1.0:1787826365.396461:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226655:x1874663332684800:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826365.396483:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826365.396506:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4c45c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b72 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.396593:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.396606:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.396623:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.396635:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.396642:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.396649:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:1.0:1787826365.396719:0:35080:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00010000:00010000:0.0:1787826365.612385:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826365.660918:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826365.660982:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826365.661017:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826365.661053:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826365.661071:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826365.661092:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826365.661466:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332685312 00000100:00100000:1.0:1787826365.661477:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332685312 00000100:00100000:1.0:1787826365.661545:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332685184 00000100:00100000:1.0:1787826365.661549:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332685184 00000100:00100000:1.0:1787826365.661575:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14384 00000100:00100000:1.0:1787826365.661585:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332685312:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826365.661679:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332685312:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 95us (696us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826365.661687:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14384 00000100:00100000:1.0:1787826365.661707:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14385 00000100:00100000:1.0:1787826365.661714:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332685184:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826365.661741:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2395:x1874663332685184:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (669us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826365.661747:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14385 00000100:00080000:1.0:1787826365.661767:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787826365.661771:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826365.665219:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332685056 00000100:00100000:0.0:1787826365.665384:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332685568 00000100:00100000:0.0:1787826365.665393:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332685568 00000100:00100000:3.0:1787826365.666060:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332685056 00000100:00100000:3.0:1787826365.666168:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332685440 00000100:00100000:3.0:1787826365.666174:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332685440 00000100:00100000:3.0:1787826365.666216:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8886 00000100:00100000:3.0:1787826365.666228:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843436d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332685056:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826365.666337:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843436d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332685056:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 111us (5431us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.666345:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8886 00000100:00100000:3.0:1787826365.666373:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8887 00000100:00100000:3.0:1787826365.666380:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332685440:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826365.666418:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332685440:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 39us (5325us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.666424:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8887 00000100:00100000:3.0:1787826365.666449:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332684928 00000100:00100000:3.0:1787826365.666453:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332684928 00000100:00100000:3.0:1787826365.666483:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2242 00000100:00100000:3.0:1787826365.666489:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784331c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332684928:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826365.666514:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784331c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332684928:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (5461us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.666520:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2242 00000100:00080000:3.0:1787826365.666693:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826365.667846:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826365.667890:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8888 00000100:00100000:1.0:1787826365.667903:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332685568:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826365.668023:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332685568:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 121us (7002us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826365.668031:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8888 00000100:00100000:2.0:1787826365.836213:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826365.837074:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332685696 00000100:00100000:0.0:1787826365.837087:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332685696 00000100:00100000:0.0:1787826365.837186:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68683 00000100:00100000:0.0:1787826365.837197:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226657:x1874663332685696:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826365.837223:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826365.837250:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c471 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b79 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.837333:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.837349:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.837365:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826365.837417:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826365.837421:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826365.837443:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.837469:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c471 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.837489:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.837497:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826365.837501:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826365.837505:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826365.837708:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226657:x1874663332685696:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 512us (1504us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826365.837715:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68683 00000100:00100000:2.0:1787826365.852150:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826365.852254:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332685824 00000100:00100000:1.0:1787826365.852265:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332685824 00000100:00100000:1.0:1787826365.852339:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68684 00000100:00100000:1.0:1787826365.852350:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226657:x1874663332685824:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826365.852373:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826365.852396:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c47f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b80 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852472:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852492:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852502:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852517:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852525:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852534:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852618:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852627:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852647:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826365.852672:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852688:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852695:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852705:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852718:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852724:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826365.852729:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852736:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852747:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826365.852750:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826365.852754:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852759:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4c486 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852771:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852777:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852782:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852797:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826365.852801:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852807:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852814:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826365.852816:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826365.852819:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.852825:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c48d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.855092:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c47f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.855144:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.855155:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:1.0:1787826365.855158:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826365.855161:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826365.855328:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226657:x1874663332685824:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 2978us (3186us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826365.855338:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68684 00000100:00100000:0.0:1787826365.861909:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826365.862012:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332685952 00000100:00100000:1.0:1787826365.862022:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332685952 00000100:00100000:1.0:1787826365.862089:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68685 00000100:00100000:1.0:1787826365.862101:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:226657:x1874663332685952:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826365.862152:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826365.862175:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c49b lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b87 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.862239:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c4a2 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.862253:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c4a2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.862278:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c4a2 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826365.862390:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5c:0x0]: returning size 4096 00000004:00200000:1.0:1787826365.862396:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826365.862451:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c49b lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b87 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.862469:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c4a2 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b87 expref: 6 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.862479:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:1.0:1787826365.862482:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826365.862486:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:1.0:1787826365.862563:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226657:x1874663332685952:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 464us (665us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826365.862570:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68685 00000100:00100000:2.0:1787826365.889265:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826365.894321:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332686080 00000100:00100000:1.0:1787826365.894340:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332686080 00000100:00100000:1.0:1787826365.894533:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68686 00000100:00100000:1.0:1787826365.894554:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226657:x1874663332686080:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:1.0:1787826365.894576:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826365.894601:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4a9 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b8e expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.894672:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.894686:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.894698:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.894713:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.894719:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.894724:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895101:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c4be lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895145:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c4be lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895153:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c4be lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826365.895208:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5d:0x0]: returning size 6 00000004:00200000:1.0:1787826365.895212:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826365.895236:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c4be lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895250:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c4be lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895261:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895279:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895287:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895294:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895301:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826365.895306:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895313:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895326:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:1.0:1787826365.895329:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826365.895334:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895344:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c4b0 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895360:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895366:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895372:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895378:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826365.895380:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895387:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895394:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x4ec94fd6 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826365.895397:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826365.895399:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895420:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c4b7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895457:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4a9 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b8e expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895478:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c4be lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b8e expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.895486:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:1.0:1787826365.895489:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826365.895491:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826365.895575:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226657:x1874663332686080:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1024us (6320us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826365.895584:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68686 00000100:00100000:2.0:1787826365.900157:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826365.900242:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332686208 00000100:00100000:0.0:1787826365.900254:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332686208 00000100:00100000:0.0:1787826365.900332:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68687 00000100:00100000:0.0:1787826365.900343:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226657:x1874663332686208:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826365.900365:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826365.900398:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c4c5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b95 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900478:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900491:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900501:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900511:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900518:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900524:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900623:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c4da lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900632:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c4da lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900639:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c4da lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826365.900680:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dc2:0x0]: returning size 6 00000004:00200000:0.0:1787826365.900684:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826365.900706:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c4da lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900721:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c4da lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900730:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900735:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900742:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900748:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900754:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826365.900758:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900764:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900776:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:0.0:1787826365.900779:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826365.900783:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 3/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900800:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c4cc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900815:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900822:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900827:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900832:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826365.900835:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900841:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900848:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f794b57ac00) 00010000:00010000:0.0:1787826365.900850:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826365.900853:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900858:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c4d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900883:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c4c5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b95 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900897:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c4da lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b95 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.900904:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f794b57a100) 00010000:00010000:0.0:1787826365.900906:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826365.900909:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826365.901765:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226657:x1874663332686208:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1418us (1613us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826365.901775:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68687 00000100:00100000:0.0:1787826365.910313:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826365.910419:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826365.910567:0:6541:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332686592 00000100:00100000:2.0:1787826365.910578:0:6541:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332686592 00000100:00100000:2.0:1787826365.910647:0:6541:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1237 00000100:00100000:2.0:1787826365.910658:0:6541:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226657:x1874663332686592:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:2.0:1787826365.910711:0:6541:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826365.910739:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c4da lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b95 expref: 12 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.910760:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f794b57a100) 00010000:00010000:2.0:1787826365.910773:0:6541:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826365.910857:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346336512:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826365.910909:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826365.910959:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f61da80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826365.911256:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346336512 00000100:00100000:0.0:1787826365.911261:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346336512 00000100:00100000:0.0:1787826365.911307:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24125 00000100:00100000:0.0:1787826365.911315:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346336512:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826365.911405:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346336512:12345-0@lo:1000:mdt_io00_001.0 Request processed in 90us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826365.911422:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24125 00000100:00100000:2.0:1787826365.911977:0:6541:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985474 gen 2 00000004:00080000:2.0:1787826365.911984:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985473 committed transno 4294985474 00000020:00080000:2.0:1787826365.911990:0:6541:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7850705300 refcount 2 committed 0 result 0 batchid 4294979089 00000020:00080000:2.0:1787826365.911994:0:6541:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7850705240 00000020:00080000:2.0:1787826365.911998:0:6541:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].946 00000020:00080000:2.0:1787826365.912179:0:6541:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abfff00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784f5f4c00 00000020:00080000:2.0:1787826365.912182:0:6541:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].943 00000004:00080000:2.0:1787826365.912211:0:6541:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985474 committed transno 4294985474 00000100:00100000:2.0:1787826365.912229:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346336512:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826365.912319:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346336640:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826365.912358:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826365.912379:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f61dc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826365.912493:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346336640 00000100:00100000:3.0:1787826365.912499:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346336640 00000100:00100000:3.0:1787826365.912554:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24126 00000100:00100000:3.0:1787826365.912563:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346336640:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.912642:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346336640:12345-0@lo:1000:mdt_io00_001.0 Request processed in 80us (283us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.912650:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24126 00000020:00080000:0.0:1787826365.913729:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979089 committed batchid 4294979088 00000020:00080000:0.0:1787826365.913814:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979088 00000100:00100000:2.0:1787826365.914599:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346336640:0@lo:1000:mdt_io00_001.0 00000004:00080000:2.0:1787826365.914647:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19c:0x0] 00000100:00100000:2.0:1787826365.914701:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346336768:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826365.914760:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826365.914803:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f61da80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826365.916370:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346336768 00000100:00100000:0.0:1787826365.916383:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346336768 00000100:00100000:0.0:1787826365.916461:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24127 00000100:00100000:0.0:1787826365.916472:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346336768:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826365.916746:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346336768:12345-0@lo:1000:mdt_io00_001.0 Request processed in 275us (1985us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826365.916754:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24127 00000100:00100000:3.0:1787826365.916959:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346336768:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826365.917197:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5c:0x0] 00000100:00100000:3.0:1787826365.917257:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346336896:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.917315:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826365.917339:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826365.917357:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346336896 00000100:00100000:3.0:1787826365.917362:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346336896 00000100:00100000:3.0:1787826365.917389:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24128 00000100:00100000:3.0:1787826365.917396:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346336896:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826365.921137:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332686336 00000100:00100000:1.0:1787826365.921153:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332686336 00000100:00100000:1.0:1787826365.921222:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11214 00000100:00100000:1.0:1787826365.921234:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332686336:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:1.0:1787826365.921248:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826365.921286:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece600/0xe21309aa81f4c4be lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b8e expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.921307:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:1.0:1787826365.921318:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826365.921371:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332686336:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 139us (11069us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826365.921378:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11214 00000100:00100000:3.0:1787826365.921708:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346336896:12345-0@lo:1000:mdt_io00_001.0 Request processed in 4309us (4389us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.921717:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24128 00000100:00100000:0.0:1787826365.925581:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346336896:0@lo:1000:mdt_io00_001.0 00000004:00080000:0.0:1787826365.925667:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19c:0x0] 00000100:00100000:0.0:1787826365.925723:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337024:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826365.925799:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826365.925952:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826365.926693:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346337024 00000100:00100000:3.0:1787826365.926703:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346337024 00000100:00100000:3.0:1787826365.926767:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24129 00000100:00100000:3.0:1787826365.926778:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346337024:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.926906:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346337024:12345-0@lo:1000:mdt_io00_001.0 Request processed in 130us (1107us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.926914:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24129 00000100:00100000:0.0:1787826365.928327:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337024:0@lo:1000:mdt_io00_001.0 00000004:00020000:0.0:1787826365.928394:0:6541:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:0.0:1787826365.928471:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.928483:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.928493:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826365.928515:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337152:0@lo:101:mdt_io00_001.0 00000004:00020000:1.0:1787826365.928665:0:35080:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4468 00010000:00010000:1.0:1787826365.928733:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928751:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928761:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826365.928772:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dc2:0x0]: returning size 6 00010000:00010000:1.0:1787826365.928875:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928883:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928888:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928893:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928899:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826365.928903:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928909:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928930:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:1.0:1787826365.928933:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826365.928937:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928946:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c463 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928960:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928965:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928971:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928975:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826365.928977:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928982:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.928989:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f794b57ac00) 00010000:00010000:1.0:1787826365.928991:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826365.928994:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.929007:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c46a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.929058:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4c45c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.929077:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.929084:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826365.929086:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826365.929088:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826365.929201:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226655:x1874663332684800:12345-192.168.203.56@tcp:101:cat.0 Request processed in 532741us (533047us total) trans 8589957486 rc 0/0 00000100:00100000:1.0:1787826365.929210:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68682 00000100:00100000:0.0:1787826365.930033:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826365.930316:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826365.935886:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346337152 00000100:00100000:2.0:1787826365.935902:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346337152 00000100:00100000:2.0:1787826365.935983:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24130 00000100:00100000:2.0:1787826365.935992:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:6541:x1874663346337152:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:2.0:1787826365.936014:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826365.936039:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c4e1 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.936076:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.936088:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 4/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c4e1 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.936147:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.936161:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 6 pid: 35093 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826365.936176:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844294a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346337280:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826365.936259:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f61da80 going to sleep for 11 seconds 00000001:00080000:1.0:1787826365.938891:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976171 is committed 00000100:00100000:3.0:1787826365.943933:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844294a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346337280:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826365.943978:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c4e1 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.943991:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826365.943994:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826365.943998:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c4e1 expref: 6 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.944009:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:3.0:1787826365.944078:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6541:x1874663346337152:12345-0@lo:101:mdt_io00_001.0 Request processed in 8086us (14160us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.944088:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24130 00000100:00100000:0.0:1787826365.944293:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11d500 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337152:0@lo:101:mdt_io00_001.0 00010000:00010000:0.0:1787826365.944315:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.944325:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826365.950403:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826365.950800:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332686720 00000100:00100000:0.0:1787826365.950815:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332686720 00010000:00010000:0.0:1787826365.950870:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 7 pid: 35093 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826365.950910:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22065 00000100:00100000:0.0:1787826365.950931:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332686720:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826365.950948:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826365.950956:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 7 pid: 35093 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.950971:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 7 pid: 35093 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.950997:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c447 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b64 expref: 7 pid: 35093 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.951018:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:0.0:1787826365.951026:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c4e1 expref: 5 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.951036:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c4e1 expref: 5 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.951045:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c4e1 expref: 5 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.951084:0:6513:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c4e1 expref: 5 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826365.951100:0:6513:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6513:1874663346337408:0@lo:105: 00000100:00100000:0.0:1787826365.951213:0:6513:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826365.951279:0:6513:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826365.951529:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346337408 00000100:00100000:2.0:1787826365.951535:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346337408 00000100:00100000:2.0:1787826365.951592:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7925 00000100:00100000:2.0:1787826365.951600:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6513:x1874663346337408:12345-0@lo:105: 00010000:00010000:2.0:1787826365.951612:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.951627:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.951636:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 5/0,1 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.951643:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.951650:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.951658:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 6/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.951688:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43800) 00000100:00100000:2.0:1787826365.951737:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6513:x1874663346337408:12345-0@lo:105: Request processed in 138us (523us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826365.951745:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7925 00000100:00100000:2.0:1787826365.951790:0:6513:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6513:1874663346337408:0@lo:105: 00010000:00010000:2.0:1787826365.951807:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826365.951856:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332686720:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 936us (1482us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826365.951864:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22065 00010000:00010000:1.0:1787826365.952142:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.952152:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826365.952177:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19c:0x0] 00010000:00010000:1.0:1787826365.952222:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.952229:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.952239:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826365.952256:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337536:0@lo:101:mdt_io00_001.0 00000100:00100000:1.0:1787826365.952314:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826365.952349:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826365.952728:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346337536 00000100:00100000:2.0:1787826365.952736:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346337536 00000100:00100000:2.0:1787826365.952789:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24131 00000100:00100000:2.0:1787826365.952798:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6541:x1874663346337536:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:2.0:1787826365.952815:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826365.952834:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c504 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c4fd expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.952857:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c504 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c4fd expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.952870:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c504 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c4fd expref: 7 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.952877:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x4ec94fd6 (ffff9f784abfee00) 00010000:00010000:2.0:1787826365.952880:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826365.952882:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826365.953422:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346337536:12345-0@lo:101:mdt_io00_001.0 Request processed in 623us (1106us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826365.953429:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24131 00000100:00100000:2.0:1787826365.957048:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826365.957168:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332686848 00000100:00100000:0.0:1787826365.957181:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332686848 00000100:00100000:0.0:1787826365.957232:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30887 00000100:00100000:0.0:1787826365.957242:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0+-99:2392:x1874663332686848:12345-192.168.203.56@tcp:8:cat.0 00010000:00080000:0.0:1787826365.957292:0:35352:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from 398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp t4294972907 exp 0000000000000000 cur 8178 last 0 00000020:00000080:0.0:1787826365.957335:0:35352:0:(genops.c:1385:class_connect()) connect: client 398e0287-d3c7-44dd-b1d9-6b85f698e615, cookie 0xe21309aa81f4c50b 00002000:00100000:0.0:1787826365.957350:0:35352:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli 398e0287-d3c7-44dd-b1d9-6b85f698e615/ffff9f784357b000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113950 ocd_grant: 8351744 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1787826365.957371:0:35352:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:0.0:1787826365.957466:0:35352:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:0.0:1787826365.957503:0:35352:0:(import.c:65:import_set_state_nolock()) ffff9f78435ba000 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1787826365.957586:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2392:x1874663332686848:12345-192.168.203.56@tcp:8:cat.0 Request processed in 343us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826365.957593:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30887 00000100:00100000:1.0:1787826365.962513:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337536:0@lo:101:mdt_io00_001.0 00010000:00010000:1.0:1787826365.962895:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c504 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.962913:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c504 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826365.962955:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19c:0x0] 00010000:00010000:1.0:1787826365.962988:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.963005:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.963015:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.963059:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826365.963073:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 10 pid: 6529 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826365.963088:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346337664:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826365.963221:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826365.976527:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826365.976688:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346337664:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826365.976722:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826365.980265:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332687104 00000100:00100000:0.0:1787826365.980291:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332687104 00010000:00010000:0.0:1787826365.980349:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 11 pid: 6529 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826365.980398:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22066 00000100:00100000:0.0:1787826365.980417:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332687104:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826365.980433:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826365.980440:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 11 pid: 6529 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.980454:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 11 pid: 6529 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.980483:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c455 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b6b expref: 11 pid: 6529 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.980501:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:0.0:1787826365.980508:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.980517:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.980524:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826365.980866:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826365.981010:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332687104:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 602us (4491us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826365.981019:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22066 00010000:00010000:2.0:1787826365.981321:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.981331:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.981353:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.981368:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826365.981387:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826365.981472:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844294700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337792:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826365.981786:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826365.981841:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f61da80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826365.982029:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346337792 00000100:00100000:0.0:1787826365.982036:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346337792 00000100:00100000:0.0:1787826365.982090:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24132 00000100:00100000:0.0:1787826365.982100:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346337792:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826365.982206:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346337792:12345-0@lo:1000:mdt_io00_001.0 Request processed in 107us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826365.982214:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24132 00000100:00100000:2.0:1787826365.983697:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844294700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337792:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826365.983785:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844294700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337920:0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826365.983851:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826365.983886:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f61dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826365.984015:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346337920 00000100:00100000:1.0:1787826365.984023:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346337920 00000100:00100000:1.0:1787826365.984074:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24133 00000100:00100000:1.0:1787826365.984084:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346337920:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826365.984201:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346337920:12345-0@lo:1000:mdt_io00_001.0 Request processed in 118us (349us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826365.984209:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24133 00000100:00100000:3.0:1787826365.984311:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844294700 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346337920:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.984359:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338048:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.984402:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826365.984430:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851ff5000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826365.984945:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346338048 00000100:00100000:2.0:1787826365.984951:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346338048 00000100:00100000:2.0:1787826365.984990:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24134 00000100:00100000:2.0:1787826365.984997:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346338048:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826365.985146:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346338048:12345-0@lo:1000:mdt_io00_001.0 Request processed in 149us (742us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826365.985153:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24134 00000100:00100000:1.0:1787826365.985297:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338048:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826365.985388:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338176:0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826365.985434:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826365.985458:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826365.985796:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346338176 00000100:00100000:3.0:1787826365.985802:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346338176 00000100:00100000:3.0:1787826365.985841:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24135 00000100:00100000:3.0:1787826365.985849:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346338176:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.985927:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346338176:12345-0@lo:1000:mdt_io00_001.0 Request processed in 69us (483us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.985934:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24135 00000100:00100000:3.0:1787826365.985966:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338176:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.986041:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338304:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.986071:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826365.986088:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826365.986099:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346338304 00000100:00100000:3.0:1787826365.986124:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346338304 00000100:00100000:3.0:1787826365.986143:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24136 00000100:00100000:3.0:1787826365.986149:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346338304:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826365.986206:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346338304:12345-0@lo:1000:mdt_io00_001.0 Request processed in 56us (134us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826365.986212:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24136 00000100:00100000:3.0:1787826365.986238:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338304:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826365.986261:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5c:0x0] 00010000:00010000:3.0:1787826365.986349:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.986383:0:6541:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.986392:0:6541:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.986990:0:6541:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826365.987006:0:6541:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 9 pid: 35080 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826365.987020:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346338432:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826365.987100:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826365.994410:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826365.994742:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332686976 00000100:00100000:0.0:1787826365.994753:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332686976 00000100:00100000:0.0:1787826365.994846:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30888 00000100:00100000:0.0:1787826365.994858:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332686976:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826365.994879:0:8443:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826365.994900:0:8443:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4c527 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801b9c expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826365.994981:0:8443:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346338560:192.168.203.56@tcp:106: 00000100:00100000:0.0:1787826365.995077:0:8443:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358e40 going to sleep for 11 seconds 00000100:00080000:2.0:1787826365.996187:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826365.999708:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000_UUID:6541:1874663346338432:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826365.999743:0:6541:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826366.001286:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826366.001380:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332687232 00000100:00100000:1.0:1787826366.001389:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332687232 00010000:00010000:1.0:1787826366.001431:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 10 pid: 35080 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826366.001469:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22067 00000100:00100000:1.0:1787826366.001478:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332687232:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826366.001492:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826366.001498:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 10 pid: 35080 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.001511:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 10 pid: 35080 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.001535:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c4e8 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b72 expref: 10 pid: 35080 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.001555:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826366.001560:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.001569:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.001576:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.001600:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826366.001646:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332687232:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 169us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.001653:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22067 00010000:00010000:3.0:1787826366.001805:0:6541:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.001815:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.001867:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.001875:0:6541:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.001886:0:6541:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826366.001905:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338688:0@lo:101:mdt_io00_001.0 00000100:00100000:3.0:1787826366.001952:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.001980:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826366.002032:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346338688 00000100:00100000:2.0:1787826366.002036:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346338688 00000100:00100000:2.0:1787826366.002072:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24137 00000100:00100000:2.0:1787826366.002079:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6541:x1874663346338688:12345-0@lo:101:mdt_io00_001.0 00010000:00010000:2.0:1787826366.002094:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826366.002148:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c535 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c52e expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826366.002167:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c535 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c52e expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826366.002177:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c535 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c52e expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826366.002185:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f784abfe600) 00010000:00010000:2.0:1787826366.002188:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1787826366.002191:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826366.002219:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346338688:12345-0@lo:101:mdt_io00_001.0 Request processed in 141us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826366.002224:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24137 00000100:00100000:3.0:1787826366.002841:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338688:0@lo:101:mdt_io00_001.0 00010000:00010000:3.0:1787826366.002858:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c535 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.002869:0:6541:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c535 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826366.002889:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5d:0x0] 00000100:00100000:3.0:1787826366.002955:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338816:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.002992:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.003023:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483340c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826366.003142:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346338816 00000100:00100000:2.0:1787826366.003147:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346338816 00000100:00100000:2.0:1787826366.003190:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24138 00000100:00100000:2.0:1787826366.003200:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346338816:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826366.003295:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346338816:12345-0@lo:1000:mdt_io00_001.0 Request processed in 96us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826366.003302:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24138 00002000:00010000:0.0:1787826366.004145:0:8443:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000401:0x523:0x0] updating lvb size: 0 -> 6 00002000:00010000:0.0:1787826366.004150:0:8443:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000401:0x523:0x0] updating lvb mtime: 0 -> 1787826360 00002000:00010000:0.0:1787826366.004152:0:8443:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000401:0x523:0x0] updating lvb atime: 0 -> 1787826360 00002000:00010000:0.0:1787826366.004154:0:8443:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000401:0x523:0x0] updating lvb ctime: 0 -> 1787826360 00002000:00010000:0.0:1787826366.004157:0:8443:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000401:0x523:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:0.0:1787826366.004175:0:8443:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8443:1874663346338560:192.168.203.56@tcp:106: 00010000:00010000:0.0:1787826366.004205:0:8443:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4c527 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b9c expref: 7 pid: 8443 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.004225:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005896499e/0xe21309aa81f4c527 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b9c expref: 7 pid: 8443 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.004236:0:8443:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:0.0:1787826366.004297:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332686976:12345-192.168.203.56@tcp:101:cat.0 Request processed in 9441us (9897us total) trans 0 rc 301/301 00000100:00100000:0.0:1787826366.004305:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30888 00000100:00100000:3.0:1787826366.006326:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338816:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826366.006378:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19c:0x0] 00000100:00100000:3.0:1787826366.006439:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338944:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.006500:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.006541:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483340c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826366.006892:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346338944 00000100:00100000:2.0:1787826366.006901:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346338944 00000100:00100000:2.0:1787826366.006971:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24139 00000100:00100000:2.0:1787826366.006982:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346338944:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:2.0:1787826366.007088:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346338944:12345-0@lo:1000:mdt_io00_001.0 Request processed in 107us (587us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826366.007096:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24139 00000100:00100000:3.0:1787826366.008712:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b7100 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346338944:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826366.008758:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5c:0x0] 00010000:00010000:3.0:1787826366.008803:0:6541:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.008818:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.008830:0:6541:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826366.008895:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339072:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.008946:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.008989:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483340c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826366.009013:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346339072 00000100:00100000:1.0:1787826366.009021:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346339072 00000100:00100000:1.0:1787826366.009070:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24140 00000100:00100000:1.0:1787826366.009081:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339072:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826366.009213:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339072:12345-0@lo:1000:mdt_io00_001.0 Request processed in 134us (265us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.009221:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24140 00000100:00100000:3.0:1787826366.009910:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339072:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826366.009951:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5c:0x0] 00000100:00100000:3.0:1787826366.009985:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339200:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.010020:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.010051:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483340c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826366.011232:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346339200 00000100:00100000:0.0:1787826366.011243:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346339200 00000100:00100000:0.0:1787826366.011311:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24141 00000100:00100000:0.0:1787826366.011323:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339200:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826366.011431:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339200:12345-0@lo:1000:mdt_io00_001.0 Request processed in 110us (1409us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826366.011439:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24141 00000100:00100000:3.0:1787826366.015720:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339200:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.015832:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339328:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.015904:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.015961:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483340c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826366.016210:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346339328 00000100:00100000:1.0:1787826366.016218:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346339328 00000100:00100000:1.0:1787826366.016290:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24142 00000100:00100000:1.0:1787826366.016300:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339328:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826366.016405:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339328:12345-0@lo:1000:mdt_io00_001.0 Request processed in 106us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.016412:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24142 00000100:00100000:0.0:1787826366.016486:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339328:0@lo:1000:mdt_io00_001.0 00000004:00080000:0.0:1787826366.016515:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19c:0x0] 00000100:00100000:0.0:1787826366.016563:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339456:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826366.016594:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826366.016618:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826366.016902:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346339456 00000100:00100000:1.0:1787826366.016907:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346339456 00000100:00100000:1.0:1787826366.016953:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24143 00000100:00100000:1.0:1787826366.016961:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339456:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826366.017024:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339456:12345-0@lo:1000:mdt_io00_001.0 Request processed in 63us (428us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.017031:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24143 00000100:00100000:0.0:1787826366.017180:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339456:0@lo:1000:mdt_io00_001.0 00000004:00080000:0.0:1787826366.017207:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5c:0x0] 00000100:00100000:0.0:1787826366.017242:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339584:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826366.017279:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826366.017294:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826366.017493:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346339584 00000100:00100000:1.0:1787826366.017498:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346339584 00000100:00100000:1.0:1787826366.017528:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24144 00000100:00100000:1.0:1787826366.017543:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339584:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:1.0:1787826366.017593:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339584:12345-0@lo:1000:mdt_io00_001.0 Request processed in 58us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.017599:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24144 00000100:00100000:0.0:1787826366.017707:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339584:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826366.017736:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339712:0@lo:1000:mdt_io00_001.0 00000100:00100000:0.0:1787826366.017758:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826366.017778:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826366.017887:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346339712 00000100:00100000:3.0:1787826366.017891:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346339712 00000100:00100000:3.0:1787826366.017929:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24145 00000100:00100000:3.0:1787826366.017936:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339712:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.017984:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339712:12345-0@lo:1000:mdt_io00_001.0 Request processed in 49us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826366.017990:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24145 00000100:00100000:3.0:1787826366.018015:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339712:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.018078:0:6541:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339840:0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.018132:0:6541:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.018153:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346339840 00000100:00100000:3.0:1787826366.018157:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346339840 00000100:00100000:3.0:1787826366.018179:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24146 00000100:00100000:3.0:1787826366.018185:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339840:12345-0@lo:1000:mdt_io00_001.0 00000100:00100000:3.0:1787826366.018235:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6541:x1874663346339840:12345-0@lo:1000:mdt_io00_001.0 Request processed in 49us (130us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826366.018241:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24146 00000100:00100000:3.0:1787826366.018258:0:6541:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483346c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826366.018281:0:6541:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465c8e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_001:lustre-MDT0000-mdtlov_UUID:6541:1874663346339840:0@lo:1000:mdt_io00_001.0 00000004:00080000:3.0:1787826366.018744:0:6541:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1dc2:0x0] is inserted into orphan list, open count = 1 00000040:00080000:3.0:1787826366.018874:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].944, 1240 off888504 00000040:00080000:3.0:1787826366.018986:0:6541:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].947, 1320 off988448 00000004:00080000:3.0:1787826366.018992:0:6541:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7848334c00 with version 1840 00000004:00080000:0.0:1787826366.019086:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1840 rpc_version 1840 00000020:00100000:0.0:1787826366.019125:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826366.019135:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d8bf480 x1874663346340096/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826366.019153:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346340096:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826366.019193:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826366.019210:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826366.019226:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346340096 00000100:00100000:0.0:1787826366.019231:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346340096 00000100:00100000:0.0:1787826366.019263:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24147 00000100:00100000:0.0:1787826366.019283:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346340096:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826366.019650:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346340096:12345-0@lo:1000:osp_up1-0.0 Request processed in 368us (455us total) trans 4294985484 rc 0/0 00000100:00100000:0.0:1787826366.019657:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24147 00000100:00100000:0.0:1787826366.019699:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346340096:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787826366.019974:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.019986:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.019995:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020001:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826366.020006:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020018:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020033:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f7967ac1d00) 00010000:00010000:3.0:1787826366.020036:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826366.020041:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020047:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c520 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826366.020068:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5d:0x0] 00010000:00010000:3.0:1787826366.020075:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c535 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020083:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c535 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020095:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020101:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020136:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020142:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826366.020145:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020156:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020164:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc200) 00010000:00010000:3.0:1787826366.020167:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826366.020169:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020175:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c512 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826366.020190:0:6541:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000175b7d96 reply state 00000000f78b15ec transno 8589957487 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020204:0:6541:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020211:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f794b57ac00) 00010000:00010000:3.0:1787826366.020213:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826366.020216:0:6541:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:3.0:1787826366.020219:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19c:0x0] 00010000:00010000:3.0:1787826366.020224:0:6541:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020231:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020238:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020265:0:6541:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826366.020279:0:6541:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19c:0x0] 00010000:00010000:3.0:1787826366.020283:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c504 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020290:0:6541:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c504 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020298:0:6541:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020303:0:6541:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020308:0:6541:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020312:0:6541:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826366.020315:0:6541:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020320:0:6541:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020327:0:6541:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f79785cc800) 00010000:00010000:3.0:1787826366.020329:0:6541:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826366.020332:0:6541:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.020336:0:6541:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c53c lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.020891:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826366.020898:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.020906:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826366.020911:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826366.020928:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.020959:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.020966:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826366.021018:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346340224:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826366.021065:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826366.021209:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.021218:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7845d43800/0xe21309aa81f4c4e1 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c4f6 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.021241:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826366.021243:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826366.021244:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826366.022195:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346340224 00000100:00100000:0.0:1787826366.022206:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346340224 00000100:00100000:0.0:1787826366.022277:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11215 00000100:00100000:0.0:1787826366.022288:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+9:3654:x1874663346340224:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826366.022302:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826366.022325:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c4f6 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c4e1 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.022342:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:0.0:1787826366.022352:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826366.024694:0:6541:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+13:226657:x1874663332686592:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 114025us (114273us total) trans 8589957487 rc 0/0 00000100:00100000:3.0:1787826366.024707:0:6541:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1237 00000100:00100000:2.0:1787826366.026305:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826366.026814:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3654:x1874663346340224:12345-0@lo:103:ldlm_bl.0 Request processed in 4524us (5744us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826366.026828:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11215 00000100:00100000:3.0:1787826366.029303:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332687360 00000100:00100000:3.0:1787826366.029320:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332687360 00000100:00100000:3.0:1787826366.029670:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68688 00000100:00100000:3.0:1787826366.029685:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226655:x1874663332687360:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826366.029711:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826366.029737:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c543 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801ba3 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.029815:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c543 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ba3 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.029831:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c543 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ba3 expref: 10 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.029851:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f79785cc700) 00010000:00010000:3.0:1787826366.029855:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787826366.029858:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68400, rc 0) 00000100:00100000:3.0:1787826366.030060:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226655:x1874663332687360:12345-192.168.203.56@tcp:101:cat.0 Request processed in 378us (3761us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826366.030068:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68688 00000100:00100000:1.0:1787826366.030173:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346340224:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826366.034376:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826366.034488:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332687488 00000100:00100000:3.0:1787826366.034498:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332687488 00000100:00100000:3.0:1787826366.034564:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30889 00000100:00100000:3.0:1787826366.034576:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226655:x1874663332687488:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826366.034595:0:20167:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826366.034614:0:20167:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801baa expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.034644:0:20167:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 5 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.034653:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x523:0x0].0x0 since=5s v=(0 0) 00010000:00010000:3.0:1787826366.034657:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826366.034662:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 4/0,0 mode: --/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 7 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.034693:0:20167:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 5 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.034706:0:20167:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 5 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826366.034720:0:20167:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346340352:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826366.034805:0:20167:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826366.037476:0:20167:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346340352:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826366.037518:0:20167:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 3/0,0 mode: --/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 7 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.037642:0:20167:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826366.037708:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226655:x1874663332687488:12345-192.168.203.56@tcp:101:cat.0 Request processed in 3133us (3340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826366.037717:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30889 00000100:00100000:0.0:1787826366.037857:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826366.038215:0:8449:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332687616 00000100:00100000:3.0:1787826366.038223:0:8449:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332687616 00002000:00100000:3.0:1787826366.038287:0:8449:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x523:0x0] (0->5) req@ffff9f784d8bea00 x1874663332687616/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:722/0 lens 496/0 e 0 to 0 dl 1787826377 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826366.038298:0:8449:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d8bea00 with x1874663332687616 ext(0->5) 00010000:00010000:3.0:1787826366.038306:0:8449:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 6 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.038320:0:8449:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 6 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826366.038328:0:8449:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9f784d8bea00 00000100:00100000:3.0:1787826366.038365:0:8449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2492 00000100:00100000:3.0:1787826366.038376:0:8449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332687616:12345-192.168.203.56@tcp:4:bash.0 00000100:00080000:0.0:1787826366.039274:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826366.039488:0:8449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332687616:12345-192.168.203.56@tcp:4:bash.0 Request processed in 1111us (1628us total) trans 4294973863 rc 0/0 00000100:00100000:3.0:1787826366.039496:0:8449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2492 00002000:00100000:3.0:1787826366.039504:0:8449:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x523:0x0] (0->5) req@ffff9f784d8bea00 x1874663332687616/t4294973863(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:722/0 lens 496/448 e 0 to 0 dl 1787826377 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826366.039519:0:8449:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d8bea00 with x1874663332687616 ext(0->5) 00010000:00010000:3.0:1787826366.039527:0:8449:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 8 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.039551:0:8449:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 8 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826366.039559:0:8449:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9f784d8bea00 00000100:00100000:0.0:1787826366.045546:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826366.045648:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332687744 00000100:00100000:1.0:1787826366.045658:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332687744 00010000:00010000:1.0:1787826366.045702:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 8 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826366.045865:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22068 00000100:00100000:1.0:1787826366.045877:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332687744:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826366.045894:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1787826366.045925:0:17812:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x523:0x0] updating lvb blocks from disk: 1 -> 8 00010000:00010000:1.0:1787826366.045933:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 8 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.045948:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 8 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.045974:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7cd0bc1/0xe21309aa81f4c3d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b3a expref: 8 pid: 8444 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.045991:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826366.045996:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826366.046017:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.046027:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.046064:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826366.046080:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:17812:1874663346340480:192.168.203.56@tcp:105: 00000100:00100000:1.0:1787826366.046206:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7851ff5000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826366.055780:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826366.055896:0:24158:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332687872 00000100:00100000:3.0:1787826366.055906:0:24158:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332687872 00002000:00100000:3.0:1787826366.055975:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_009: refresh rw locks for [0x280000401:0x523:0x0] (0->4095) req@ffff9f784d98f800 x1874663332687872/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:722/0 lens 488/0 e 0 to 0 dl 1787826377 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826366.055987:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d98f800 with x1874663332687872 ext(0->4095) 00002000:00010000:3.0:1787826366.055993:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784d98f800 00000100:00100000:3.0:1787826366.056027:0:24158:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27667 00000100:00100000:3.0:1787826366.056039:0:24158:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332687872:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:3.0:1787826366.056232:0:24158:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332687872:12345-192.168.203.56@tcp:3:cat.0 Request processed in 195us (462us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1787826366.056240:0:24158:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27667 00002000:00100000:3.0:1787826366.056247:0:24158:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_009: refresh rw locks for [0x280000401:0x523:0x0] (0->4095) req@ffff9f784d98f800 x1874663332687872/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:722/0 lens 488/4536 e 0 to 0 dl 1787826377 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787826366.056255:0:24158:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d98f800 with x1874663332687872 ext(0->4095) 00002000:00010000:3.0:1787826366.056258:0:24158:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f784d98f800 00000100:00100000:1.0:1787826366.056640:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:17812:1874663346340480:192.168.203.56@tcp:105: 00010000:00010000:1.0:1787826366.056666:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826366.056706:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332687744:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 10831us (11170us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.056716:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22068 00000100:00100000:2.0:1787826366.061087:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826366.061968:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332688128 00000100:00100000:3.0:1787826366.061980:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332688128 00000100:00100000:3.0:1787826366.062568:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9396 00000100:00100000:3.0:1787826366.062581:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967aeec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226655:x1874663332688128:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:3.0:1787826366.062831:0:6530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1dc2:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1787826366.062926:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].217, 64 off22080 00000004:00080000:3.0:1787826366.062940:0:6530:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8178274833551ns, next at 8179274830284ns, rc = 0 00010000:00010000:3.0:1787826366.062997:0:6530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.064142:0:6530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.064156:0:6530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.064166:0:6530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.064173:0:6530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.064180:0:6530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.064322:0:6530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826366.064490:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967aeec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226655:x1874663332688128:12345-192.168.203.56@tcp:35:cat.0 Request processed in 1908us (3496us total) trans 8589957488 rc 0/0 00000100:00100000:0.0:1787826366.064500:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9396 00010000:00010000:2.0:1787826366.065061:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826366.065070:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826366.065079:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826366.065083:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826366.065092:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826366.065133:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f79785cc700) 00010000:00010000:2.0:1787826366.065136:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826366.065140:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826366.065146:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c551 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dc2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826366.065159:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826366.065161:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826366.065163:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826366.255949:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c504 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826366.255975:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c504 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.256031:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c504 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.256039:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826366.256123:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346340608:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826366.256199:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826366.256297:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346340608 00000100:00100000:1.0:1787826366.256305:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346340608 00000100:00100000:1.0:1787826366.256376:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11216 00000100:00100000:1.0:1787826366.256386:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c81450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3655:x1874663346340608:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787826366.256400:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826366.256424:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee800/0xe21309aa81f4c504 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c4fd expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.256442:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x4ec94fd6 (ffff9f784abfee00) 00010000:00010000:1.0:1787826366.256455:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826366.257099:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecfc00/0xe21309aa81f4c4fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19c:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c504 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257135:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c535 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826366.257145:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c535 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257170:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c535 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257176:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826366.257192:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e69c00/0xe21309aa81f4c52e lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c535 expref: -99 pid: 6541 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787826366.257206:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957488 is committed 00000004:00080000:0.0:1787826366.257212:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 216 committed 00000001:00080000:0.0:1787826366.257236:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957487 is committed 00000020:00080000:0.0:1787826366.257247:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979089 batchid updated 00000001:00080000:0.0:1787826366.257257:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957485 is committed 00000100:00100000:0.0:1787826366.257298:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826366.257312:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346340736:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826366.257356:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826366.257393:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257403:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257410:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257417:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826366.257422:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257429:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257440:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f794b57ac00) 00010000:00010000:0.0:1787826366.257444:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826366.257448:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257454:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c3bb lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787826366.257474:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979089 00000004:00080000:0.0:1787826366.257535:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f784a358e40 with version 1841 00000020:00080000:0.0:1787826366.257546:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979089 cancel update log [0x240000404:0x2:0x0].946: rc = 0 00000020:00080000:0.0:1787826366.257632:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979089 cancel update log [0x200000bd0:0x2:0x0].943: rc = 0 00000020:00080000:0.0:1787826366.257644:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979089 00000004:00080000:0.0:1787826366.257653:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1841 rpc_version 1841 00000020:00100000:0.0:1787826366.257673:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826366.257682:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f797c321f80 x1874663346340992/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826366.257698:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321f80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346340992:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826366.257732:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826366.257750:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784a358c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826366.257766:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346340736 00000100:00100000:0.0:1787826366.257771:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346340736 00000100:00100000:0.0:1787826366.257812:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11217 00000100:00100000:0.0:1787826366.257821:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3653:x1874663346340736:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826366.257831:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826366.257846:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c535 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c52e expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826366.257859:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f784abfe600) 00010000:00010000:0.0:1787826366.257867:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826366.257900:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1874663346340736:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 79us (542us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826366.257907:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11217 00000100:00100000:0.0:1787826366.257953:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346340736:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826366.257978:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346340992 00000100:00100000:0.0:1787826366.257981:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346340992 00010000:00010000:3.0:1787826366.258136:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258144:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258150:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258155:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826366.258159:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258165:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258174:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f794b57ac00) 00010000:00010000:3.0:1787826366.258177:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826366.258180:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258185:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c519 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6541 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826366.258301:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826366.258310:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346341120:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787826366.258344:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.258377:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346341120 00000100:00100000:3.0:1787826366.258382:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346341120 00000100:00100000:3.0:1787826366.258427:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30890 00000100:00100000:3.0:1787826366.258434:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346341120:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1787826366.258456:0:8759:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1315 count 1 00010000:00010000:3.0:1787826366.258478:0:8759:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258496:0:8759:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258510:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826366.258514:0:8759:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826366.258519:0:8759:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258544:0:8759:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 6 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.258557:0:8759:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 6 pid: 20167 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826366.258567:0:8759:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346341248:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826366.258630:0:8759:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483340c0 going to sleep for 11 seconds 00000100:00080000:3.0:1787826366.258641:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826366.258774:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24148 00000100:00100000:3.0:1787826366.258783:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:30909:x1874663346340992:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826366.258920:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:30909:x1874663346340992:12345-0@lo:1000:osp_up1-0.0 Request processed in 136us (1184us total) trans 4294985485 rc 0/0 00000100:00100000:3.0:1787826366.258927:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24148 00000100:00100000:3.0:1787826366.258979:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321f80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346340992:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826366.261445:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c81450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346340608:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 5051us (5235us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.261459:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11216 00000100:00100000:3.0:1787826366.263415:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826366.263507:0:8759:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8759:1874663346341248:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826366.263549:0:8759:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826366.263849:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332688256 00000100:00100000:1.0:1787826366.263857:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332688256 00010000:00010000:1.0:1787826366.263897:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 7 pid: 20167 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826366.263938:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22069 00000100:00100000:1.0:1787826366.263949:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2395:x1874663332688256:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826366.263963:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826366.263969:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 7 pid: 20167 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.263982:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 7 pid: 20167 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.264016:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076034fbc/0xe21309aa81f4c54a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x523:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801baa expref: 7 pid: 20167 timeout: 8278 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.264032:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826366.264037:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x523:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826366.264042:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.264053:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.264082:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826366.264155:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332688256:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 207us (760us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.264162:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22069 00010000:00010000:3.0:1787826366.264742:0:8759:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.264757:0:8759:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.264767:0:8759:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.264774:0:8759:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826366.264799:0:8759:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826366.264987:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346341120:12345-0@lo:6:osp-syn-0-0.0 Request processed in 6551us (6639us total) trans 4294973864 rc 0/0 00000100:00100000:3.0:1787826366.264995:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30890 00000004:00080000:0.0:1787826366.265148:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784d816d80/2, rc 0, transno 4294973864 00000100:00100000:0.0:1787826366.265155:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346341120:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1787826366.265243:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826366.265250:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.265260:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826366.265264:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.265289:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.265303:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.265311:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37cd113/0xe21309aa81f4c558 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x523:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8759 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826366.265332:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826366.265334:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826366.265335:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826366.265602:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346340608:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826366.444338:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826366.444457:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346341504:0@lo:400:kworker.0 00000100:00100000:2.0:1787826366.444537:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826366.444582:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346341376:0@lo:400:kworker.0 00000100:00100000:2.0:1787826366.444612:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.444722:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346341504 00000100:00100000:3.0:1787826366.444731:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346341504 00000100:00100000:3.0:1787826366.444788:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24149 00000100:00100000:3.0:1787826366.444799:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346341504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826366.444862:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346341504:12345-0@lo:400:kworker.0 Request processed in 65us (324us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826366.444869:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24149 00000100:00100000:0.0:1787826366.445063:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346341376 00000100:00100000:0.0:1787826366.445068:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346341376 00000100:00100000:0.0:1787826366.445131:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2243 00000100:00100000:0.0:1787826366.445139:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346341376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826366.445177:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346341376:12345-0@lo:400:kworker.0 Request processed in 39us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826366.445183:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2243 00000100:00100000:2.0:1787826366.445312:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957485 gen 2 00000004:00080000:2.0:1787826366.445319:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957483 committed transno 8589957485 00000004:00080000:2.0:1787826366.445336:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957484 committed transno 8589957485 00000004:00080000:2.0:1787826366.445347:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957485 committed transno 8589957485 00000100:00100000:2.0:1787826366.445362:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346341504:0@lo:400:kworker.0 00000100:00100000:2.0:1787826366.445374:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346341376:0@lo:400:kworker.0 00000100:00100000:0.0:1787826366.446441:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346341632:0@lo:400:kworker.0 00000100:00100000:0.0:1787826366.446496:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.446569:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346341632 00000100:00100000:3.0:1787826366.446576:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346341632 00000100:00100000:3.0:1787826366.446627:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14386 00000100:00100000:3.0:1787826366.446636:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346341632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826366.451910:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346341632:12345-0@lo:400:kworker.0 Request processed in 5265us (5403us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826366.451926:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14386 00000100:00100000:3.0:1787826366.452129:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346341760:0@lo:400:kworker.0 00000100:00100000:3.0:1787826366.452194:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826366.452240:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346341760 00000100:00100000:3.0:1787826366.452248:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346341760 00000100:00100000:3.0:1787826366.452310:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14387 00000100:00100000:3.0:1787826366.452319:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346341760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826366.452372:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346341760:12345-0@lo:400:kworker.0 Request processed in 54us (177us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826366.452378:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14387 00000100:00080000:3.0:1787826366.453062:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826366.453122:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346341760:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.463698:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346341888:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.463774:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826366.463829:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346341888 00000100:00100000:1.0:1787826366.463836:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346341888 00000100:00100000:1.0:1787826366.463891:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14388 00000100:00100000:1.0:1787826366.463902:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346341888:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.463953:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346341888:12345-0@lo:400:kworker.0 Request processed in 54us (179us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.463960:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14388 00000100:00080000:1.0:1787826366.463982:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826366.464012:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346341888:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.464047:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826366.464053:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346342016:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.464084:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826366.464135:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346342016 00000100:00100000:1.0:1787826366.464139:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346342016 00000100:00100000:1.0:1787826366.464175:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8889 00000100:00100000:1.0:1787826366.464183:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796eda6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346342016:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.464219:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796eda6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346342016:12345-0@lo:400:kworker.0 Request processed in 37us (134us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.464225:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8889 00000100:00080000:1.0:1787826366.466599:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826366.466655:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346342016:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.466721:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826366.466733:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346342144:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.466792:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826366.466833:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346342144 00000100:00100000:1.0:1787826366.466839:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346342144 00000100:00100000:1.0:1787826366.466889:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14389 00000100:00100000:1.0:1787826366.466899:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346342144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.466943:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346342144:12345-0@lo:400:kworker.0 Request processed in 46us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.466950:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14389 00000100:00080000:1.0:1787826366.466967:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826366.466985:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346342144:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.467014:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346342272:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.467038:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826366.467082:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346342400:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.467126:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826366.474016:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346342400 00000100:00100000:1.0:1787826366.474031:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346342400 00000100:00100000:1.0:1787826366.474140:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8890 00000100:00080000:0.0:1787826366.474382:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826366.474441:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346342400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826366.474643:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346341632:0@lo:400:kworker.0 00000100:00100000:2.0:1787826366.474775:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346342272 00000100:00100000:2.0:1787826366.474780:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346342272 00000100:00100000:2.0:1787826366.474823:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14390 00000100:00100000:2.0:1787826366.474831:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346342272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826366.474890:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346342272:12345-0@lo:400:kworker.0 Request processed in 59us (7851us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826366.474896:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14390 00000100:00080000:2.0:1787826366.474916:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826366.474939:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346342272:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.475349:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346342400:12345-0@lo:400:kworker.0 Request processed in 916us (8218us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.475359:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8890 00000100:00100000:1.0:1787826366.475446:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826366.475458:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346342656:0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.475503:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826366.475549:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346342656 00000100:00100000:1.0:1787826366.475554:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346342656 00000100:00100000:1.0:1787826366.475582:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14391 00000100:00100000:1.0:1787826366.475589:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346342656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.475624:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b662680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346342656:12345-0@lo:400:kworker.0 Request processed in 35us (121us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.475630:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14391 00000100:00100000:1.0:1787826366.475655:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b663800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346342656:0@lo:400:kworker.0 00000100:00100000:2.0:1787826366.478217:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346342528:0@lo:400:kworker.0 00000100:00100000:2.0:1787826366.478311:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826366.483166:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346342528 00000100:00100000:1.0:1787826366.483182:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346342528 00000100:00100000:1.0:1787826366.483274:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8891 00000100:00100000:1.0:1787826366.483286:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346342528:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826366.483396:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346342528:12345-0@lo:400:kworker.0 Request processed in 111us (5084us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.483554:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8891 00000100:00080000:1.0:1787826366.483581:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826366.483619:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976171 gen 2 00000004:00080000:1.0:1787826366.483625:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784e760a80, transno 4294976171 00000100:00100000:1.0:1787826366.483640:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785124f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346342528:0@lo:400:kworker.0 00000100:00100000:0.0:1787826366.484632:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346342400:0@lo:400:kworker.0 00010000:00010000:0.0:1787826366.637317:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826366.687462:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826366.687519:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826366.687579:0:24220:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332688384 00000100:00100000:2.0:1787826366.687590:0:24220:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332688384 00000100:00100000:1.0:1787826366.687645:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332688512 00000100:00100000:1.0:1787826366.687650:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332688512 00002000:00100000:2.0:1787826366.687650:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000400:0x4ce:0x0] (0->5) req@ffff9f784d816680 x1874663332688384/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:722/0 lens 496/0 e 0 to 0 dl 1787826377 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826366.687660:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d816680 with x1874663332688384 ext(0->5) 00002000:00010000:2.0:1787826366.687666:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784d816680 00000100:00100000:1.0:1787826366.687694:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30891 00000100:00100000:2.0:1787826366.687694:0:24220:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27668 00000100:00100000:2.0:1787826366.687703:0:24220:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332688384:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:1.0:1787826366.687704:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332688512:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:1.0:1787826366.687791:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2395:x1874663332688512:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 88us (339us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826366.687799:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30891 00000100:00100000:2.0:1787826366.693362:0:24220:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332688384:12345-192.168.203.56@tcp:4:bash.0 Request processed in 5645us (5828us total) trans 4294976172 rc 0/0 00000100:00100000:2.0:1787826366.693380:0:24220:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27668 00002000:00100000:2.0:1787826366.693395:0:24220:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_012: refresh rw locks for [0x2c0000400:0x4ce:0x0] (0->5) req@ffff9f784d816680 x1874663332688384/t4294976172(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:722/0 lens 496/448 e 0 to 0 dl 1787826377 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787826366.693406:0:24220:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784d816680 with x1874663332688384 ext(0->5) 00002000:00010000:2.0:1787826366.693412:0:24220:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784d816680 00000100:00100000:3.0:1787826366.957658:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826366.957684:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346342784:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826366.957770:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826366.959398:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346342784 00000100:00100000:0.0:1787826366.959411:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346342784 00000100:00100000:1.0:1787826366.959651:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6267 00000100:00100000:1.0:1787826366.959664:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346342784:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826366.959689:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826366.959778:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346342784:12345-0@lo:13:osp-pre-1-1.0 Request processed in 117us (2008us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826366.959904:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c322a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346342784:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826366.959954:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6267 00000100:00100000:0.0:1787826367.205878:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.212272:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332688640 00000100:00100000:1.0:1787826367.212288:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332688640 00000100:00100000:1.0:1787826367.212480:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68689 00000100:00100000:1.0:1787826367.212495:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226693:x1874663332688640:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826367.212520:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.212546:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c55f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bb1 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.212625:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c566 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.212640:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c566 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.212655:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c566 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826367.212767:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5c:0x0]: returning size 4096 00000004:00200000:1.0:1787826367.212772:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787826367.212793:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c566 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.212823:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c55f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb1 expref: 8 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.212845:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c566 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb1 expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.212854:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:1.0:1787826367.212857:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826367.212861:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826367.212933:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226693:x1874663332688640:12345-192.168.203.56@tcp:101:rm.0 Request processed in 443us (7064us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.212941:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68689 00000100:00080000:2.0:1787826367.216715:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826367.225029:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.226150:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332688768 00000100:00100000:1.0:1787826367.226163:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332688768 00000100:00100000:1.0:1787826367.226224:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68690 00000100:00100000:1.0:1787826367.226235:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226693:x1874663332688768:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826367.226256:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.226293:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c56d lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bb8 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.226507:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.226521:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.226532:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826367.226578:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19c:0x0]: returning size 4096 00000004:00200000:1.0:1787826367.226584:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826367.226621:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c56d lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 9 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.226638:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.226647:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f7967ac1300) 00010000:00010000:1.0:1787826367.226650:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826367.226654:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826367.226723:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226693:x1874663332688768:12345-192.168.203.56@tcp:101:rm.0 Request processed in 489us (1701us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.227443:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68690 00000100:00100000:1.0:1787826367.236981:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.237221:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332688896 00000100:00100000:3.0:1787826367.237232:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332688896 00000100:00100000:3.0:1787826367.237573:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68691 00000100:00100000:3.0:1787826367.237586:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226693:x1874663332688896:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826367.237610:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826367.237634:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c57b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bbf expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.237699:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.237713:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.237822:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826367.237870:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x19a:0x0]: returning size 4096 00000004:00200000:3.0:1787826367.237875:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826367.237912:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c57b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 11 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.237929:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.237938:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc800) 00010000:00010000:3.0:1787826367.237941:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826367.237945:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00080000:1.0:1787826367.238128:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826367.238172:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b663800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226693:x1874663332688896:12345-192.168.203.56@tcp:101:rm.0 Request processed in 589us (1200us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.238179:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68691 00000100:00100000:3.0:1787826367.255444:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.255635:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332689152 00000100:00100000:0.0:1787826367.255645:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332689152 00000100:00100000:0.0:1787826367.255711:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68692 00000100:00100000:0.0:1787826367.255722:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226693:x1874663332689152:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826367.255898:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826367.255926:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c589 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bc6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826367.256240:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5c:0x0]: returning size 4096 00010000:00010000:0.0:1787826367.256360:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c589 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bc6 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.256374:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f784b4a5500) 00010000:00010000:0.0:1787826367.256377:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826367.256382:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c589 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bc6 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.256399:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43c00, rc 0) 00000100:00100000:0.0:1787826367.256459:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854624e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226693:x1874663332689152:12345-192.168.203.56@tcp:101:rm.0 Request processed in 739us (1022us total) trans 4294985486 rc 301/301 00000100:00100000:0.0:1787826367.256467:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68692 00000100:00100000:0.0:1787826367.261364:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.261444:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332689280 00000100:00100000:2.0:1787826367.261455:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332689280 00000100:00100000:2.0:1787826367.261509:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9397 00000100:00100000:2.0:1787826367.261520:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226693:x1874663332689280:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:0.0:1787826367.265945:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226693:x1874663332689280:12345-192.168.203.56@tcp:37:rm.0 Request processed in 4417us (4591us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.265959:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9397 00000100:00100000:1.0:1787826367.278930:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.279002:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332689408 00000100:00100000:0.0:1787826367.279012:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332689408 00000100:00100000:0.0:1787826367.279079:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9398 00000100:00100000:0.0:1787826367.279089:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226693:x1874663332689408:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:0.0:1787826367.289028:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226693:x1874663332689408:12345-192.168.203.56@tcp:37:rm.0 Request processed in 9927us (10095us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.289138:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9398 00000100:00100000:0.0:1787826367.296824:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.297005:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332689664 00000100:00100000:1.0:1787826367.297017:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332689664 00000100:00100000:1.0:1787826367.297080:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9399 00000100:00100000:1.0:1787826367.297091:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226693:x1874663332689664:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:0.0:1787826367.297657:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1232, 344 off1125176 00000040:00080000:0.0:1787826367.297715:0:17601:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1550, 424 off1569472 00000004:00080000:0.0:1787826367.297722:0:17601:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7851ff53c0 with version 6892 00000004:00080000:3.0:1787826367.300159:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6892 rpc_version 6892 00000020:00100000:3.0:1787826367.300210:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826367.300222:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7854626d80 x1874663346343040/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826367.300249:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346343040:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826367.300883:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826367.301086:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826367.306174:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346343040 00000100:00100000:0.0:1787826367.306191:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346343040 00000100:00100000:0.0:1787826367.306306:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24150 00000100:00100000:0.0:1787826367.306319:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346343040:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826367.306626:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346343040:12345-0@lo:1000:osp_up0-1.0 Request processed in 309us (6282us total) trans 8589957489 rc 0/0 00000100:00100000:0.0:1787826367.306634:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24150 00000100:00100000:0.0:1787826367.313164:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346343040:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826367.316319:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226693:x1874663332689664:12345-192.168.203.56@tcp:35:rm.0 Request processed in 19219us (19494us total) trans 4294985487 rc 0/0 00000100:00100000:1.0:1787826367.316333:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9399 00000100:00100000:0.0:1787826367.330210:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.330515:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332689920 00000100:00100000:1.0:1787826367.330525:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332689920 00000100:00100000:1.0:1787826367.330595:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68693 00000100:00100000:1.0:1787826367.330606:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226693:x1874663332689920:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826367.330634:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.330658:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c597 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bcd expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.330845:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4c59e lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.330859:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4c59e lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.330870:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4c59e lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826367.330882:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5c:0x0]: returning size 4096 00010000:00010000:1.0:1787826367.332163:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c597 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bcd expref: 13 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.332196:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4c59e lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bcd expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.332211:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:1.0:1787826367.332215:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1787826367.332218:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826367.332310:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226693:x1874663332689920:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1704us (2108us total) trans 4294985488 rc 0/0 00000100:00100000:1.0:1787826367.332319:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68693 00000100:00100000:2.0:1787826367.342674:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.346366:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332690048 00000100:00100000:0.0:1787826367.346380:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332690048 00000100:00100000:0.0:1787826367.346447:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68694 00000100:00100000:0.0:1787826367.346458:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226693:x1874663332690048:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:0.0:1787826367.346516:0:24703:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345647616 00000100:00100000:0.0:1787826367.347009:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226693:x1874663332690048:12345-192.168.203.56@tcp:41:rm.0 Request processed in 550us (4342us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.347017:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68694 00000100:00100000:1.0:1787826367.353177:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.353439:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332690176 00000100:00100000:3.0:1787826367.353451:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332690176 00000100:00100000:3.0:1787826367.353517:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68695 00000100:00100000:3.0:1787826367.353537:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226693:x1874663332690176:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826367.353559:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826367.353582:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c5ac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bd4 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353658:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353672:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353683:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353698:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353704:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353711:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353786:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c5c1 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353794:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c5c1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353804:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c5c1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826367.353826:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c5c1 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353841:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353847:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353853:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353858:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353864:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826367.353868:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353875:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353887:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc800) 00010000:00010000:3.0:1787826367.353890:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826367.353894:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353900:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c5b3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353915:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353921:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353927:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353932:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826367.353935:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353940:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353947:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f79785cc200) 00010000:00010000:3.0:1787826367.353950:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826367.353963:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 2/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.353970:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4c5ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.354002:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c5ac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bd4 expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.354016:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c5c1 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bd4 expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.354027:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826367.354030:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826367.354032:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826367.354126:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967911f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226693:x1874663332690176:12345-192.168.203.56@tcp:101:rm.0 Request processed in 580us (938us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.354133:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68695 00000100:00100000:1.0:1787826367.369125:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.369175:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332690304 00000100:00100000:1.0:1787826367.369184:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332690304 00000100:00100000:1.0:1787826367.369245:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68696 00000100:00100000:1.0:1787826367.369257:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226693:x1874663332690304:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826367.369296:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.369322:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c5c8 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bdb expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.369402:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5cf lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.369417:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5cf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.369428:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5cf lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826367.369477:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5d:0x0]: returning size 6 00000004:00200000:1.0:1787826367.369482:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:1.0:1787826367.369520:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c5c8 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bdb expref: 14 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.369539:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5cf lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bdb expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.369548:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:1.0:1787826367.369551:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826367.369555:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:1.0:1787826367.369612:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967910000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226693:x1874663332690304:12345-192.168.203.56@tcp:101:rm.0 Request processed in 358us (513us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.369619:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68696 00000100:00100000:1.0:1787826367.380487:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.380533:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.380566:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332690688 00000100:00100000:1.0:1787826367.380575:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332690688 00000100:00100000:1.0:1787826367.380632:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68697 00000100:00100000:1.0:1787826367.380643:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226693:x1874663332690688:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826367.380701:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826367.381266:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5cf lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bdb expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.381286:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:1.0:1787826367.381298:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826367.381353:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.381362:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.381450:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826367.381486:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343168:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1787826367.381549:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.381578:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826367.382201:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332690432 00000100:00100000:3.0:1787826367.382209:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332690432 00000100:00100000:3.0:1787826367.382289:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11218 00000100:00100000:3.0:1787826367.382298:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332690432:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826367.382311:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826367.382328:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c5c1 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bd4 expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.383507:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346343168 00000100:00100000:2.0:1787826367.383516:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346343168 00000100:00100000:2.0:1787826367.383563:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24151 00000100:00100000:2.0:1787826367.383572:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:17813:x1874663346343168:12345-0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826367.383594:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826367.383615:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c5d6 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.383640:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.383651:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 4/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c5d6 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.383683:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.383696:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 10 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.383710:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346343296:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826367.383871:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc180 going to sleep for 11 seconds 00010000:00010000:3.0:1787826367.384181:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f79785ccd00) 00010000:00010000:3.0:1787826367.384194:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826367.384227:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332690432:12345-192.168.203.56@tcp:103:rm.0 Request processed in 1929us (3751us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.384237:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11218 00000100:00100000:1.0:1787826367.398204:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.398359:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346343296:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826367.398403:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 3/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c5d6 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.398417:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc800) 00010000:00010000:2.0:1787826367.398420:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826367.398425:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 3/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c5d6 expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.398436:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826367.398499:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:17813:x1874663346343168:12345-0@lo:101:mdt00_005.0 Request processed in 14928us (16949us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.398509:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24151 00000100:00100000:2.0:1787826367.398574:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f796cdb0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343168:0@lo:101:mdt00_005.0 00010000:00010000:2.0:1787826367.398594:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.398605:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826367.398843:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332690816 00000100:00100000:3.0:1787826367.398851:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332690816 00010000:00010000:3.0:1787826367.398886:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 10 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826367.398933:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22070 00000100:00100000:3.0:1787826367.398943:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332690816:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826367.398966:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826367.398974:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 10 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.398987:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 10 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.399009:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c582 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bbf expref: 10 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.399022:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc800) 00010000:00010000:3.0:1787826367.399027:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c5d6 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.399037:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c5d6 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.399046:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c5d6 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.399079:0:125406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c5d6 expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826367.399092:0:125406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346343424:0@lo:105: 00000100:00100000:3.0:1787826367.399154:0:125406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826367.399184:0:125406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483349c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.399750:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346343424 00000100:00100000:1.0:1787826367.399757:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346343424 00000100:00100000:1.0:1787826367.399802:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7926 00000100:00100000:1.0:1787826367.399811:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346343424:12345-0@lo:105: 00010000:00010000:1.0:1787826367.399821:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.399832:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.399841:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 5/0,1 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.399848:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.399856:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.399864:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.399894:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecfe00) 00000100:00100000:1.0:1787826367.399946:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:125406:x1874663346343424:12345-0@lo:105: Request processed in 136us (791us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.399964:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7926 00010000:00010000:0.0:1787826367.400327:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.400347:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826367.400384:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19a:0x0] 00010000:00010000:0.0:1787826367.400424:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.400433:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.400443:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.400461:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343552:0@lo:101:mdt00_005.0 00000100:00100000:0.0:1787826367.400500:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.400538:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.400699:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346343552 00000100:00100000:1.0:1787826367.400705:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346343552 00000100:00100000:1.0:1787826367.400742:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24152 00000100:00100000:1.0:1787826367.400750:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:17813:x1874663346343552:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826367.400765:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.400780:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5eb lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c5e4 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.400802:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5eb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c5e4 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.400816:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5eb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c5e4 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.400825:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f7967ac1a00) 00010000:00010000:1.0:1787826367.400828:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826367.400831:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecea00, rc 0) 00000100:00100000:1.0:1787826367.400863:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346343552:12345-0@lo:101:mdt00_005.0 Request processed in 113us (363us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.400869:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24152 00000100:00100000:3.0:1787826367.401025:0:125406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:125406:1874663346343424:0@lo:105: 00010000:00010000:3.0:1787826367.401041:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826367.401092:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332690816:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2148us (2897us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.401101:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22070 00000100:00100000:0.0:1787826367.401690:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343552:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826367.401710:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c5eb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.401722:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5eb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826367.401740:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19a:0x0] 00000100:00100000:0.0:1787826367.401782:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343680:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826367.401822:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.401846:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826367.401968:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346343680 00000100:00100000:3.0:1787826367.401974:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346343680 00000100:00100000:3.0:1787826367.402017:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24153 00000100:00100000:3.0:1787826367.402026:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346343680:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787826367.402126:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346343680:12345-0@lo:1000:mdt00_005.0 Request processed in 80us (282us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.402134:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24153 00000100:00100000:1.0:1787826367.402254:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189d880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343680:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826367.402318:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343808:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826367.402361:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.402390:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb73c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826367.402570:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346343808 00000100:00100000:0.0:1787826367.402576:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346343808 00000100:00100000:0.0:1787826367.402613:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24154 00000100:00100000:0.0:1787826367.402621:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346343808:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826367.402693:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346343808:12345-0@lo:1000:mdt00_005.0 Request processed in 73us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.402703:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24154 00000100:00100000:0.0:1787826367.402734:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343808:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826367.402783:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343936:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787826367.402810:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.402823:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.402922:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346343936 00000100:00100000:1.0:1787826367.402927:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346343936 00000100:00100000:1.0:1787826367.402968:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24155 00000100:00100000:1.0:1787826367.402976:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346343936:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787826367.403048:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346343936:12345-0@lo:1000:mdt00_005.0 Request processed in 73us (237us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826367.403055:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24155 00000100:00100000:0.0:1787826367.404036:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346343936:0@lo:1000:mdt00_005.0 00010000:00010000:0.0:1787826367.404146:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.404157:0:17813:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.404167:0:17813:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.404182:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346344064:0@lo:101:mdt00_005.0 00000100:00100000:0.0:1787826367.404227:0:17813:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.404252:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.404543:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346344064 00000100:00100000:1.0:1787826367.404549:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346344064 00000100:00100000:1.0:1787826367.404593:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24156 00000100:00100000:1.0:1787826367.404601:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:17813:x1874663346344064:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787826367.404614:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.404631:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c5f9 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c5f2 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.404655:0:6533:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c5f9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c5f2 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.404667:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c5f9 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c5f2 expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.404676:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:1.0:1787826367.404678:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787826367.404682:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826367.404728:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:17813:x1874663346344064:12345-0@lo:101:mdt00_005.0 Request processed in 127us (500us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.404735:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24156 00000100:00100000:0.0:1787826367.404861:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:17813:1874663346344064:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787826367.404877:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.404889:0:17813:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.404913:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.404927:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.404935:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787826367.406054:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].226, 64 off22656 00000004:00080000:0.0:1787826367.406073:0:17813:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 8179617966685ns, next at 8180617962270ns, rc = 0 00000040:00080000:0.0:1787826367.406169:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1233, 384 off1125560 00000040:00080000:0.0:1787826367.406226:0:17813:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1551, 464 off1569936 00000004:00080000:0.0:1787826367.406232:0:17813:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7848362000 with version 6893 00000004:00080000:3.0:1787826367.406297:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6893 rpc_version 6893 00000020:00100000:3.0:1787826367.406333:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826367.406475:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7854626300 x1874663346344320/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826367.406501:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346344320:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826367.406655:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826367.406723:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483349c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826367.406769:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346344320 00000100:00100000:0.0:1787826367.406776:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346344320 00000100:00100000:0.0:1787826367.406865:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24157 00000100:00100000:0.0:1787826367.406877:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346344320:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826367.407212:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346344320:12345-0@lo:1000:osp_up0-1.0 Request processed in 336us (556us total) trans 8589957490 rc 0/0 00000100:00100000:0.0:1787826367.407220:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24157 00000100:00100000:3.0:1787826367.407328:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7854626300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346344320:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787826367.407388:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407404:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407416:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407426:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407433:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407439:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407456:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826367.407476:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407482:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407488:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407494:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826367.407499:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407507:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407519:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:3.0:1787826367.407522:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826367.407536:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407547:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42800/0xe21309aa81f4c600 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407560:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407569:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407576:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407592:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787826367.407604:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19a:0x0] 00010000:00010000:3.0:1787826367.407612:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407620:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407628:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407641:0:17813:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787826367.407644:0:17813:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19a:0x0] 00010000:00010000:3.0:1787826367.407648:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c5eb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407661:0:17813:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c5eb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826367.407736:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226693:x1874663332690688:12345-192.168.203.56@tcp:36:rm.0 Request processed in 27092us (27199us total) trans 4294985489 rc 0/0 00000100:00100000:3.0:1787826367.407744:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68697 00010000:00010000:3.0:1787826367.407772:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826367.407777:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407784:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826367.407787:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407793:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407802:0:30445:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:3.0:1787826367.407804:0:30445:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787826367.407807:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407812:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69200/0xe21309aa81f4c607 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e5d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407827:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826367.407831:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407842:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826367.407846:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826367.407853:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407880:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407887:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826367.407910:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407916:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42000/0xe21309aa81f4c5f2 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c5f9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407929:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826367.407932:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407939:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826367.407942:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826367.407948:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407967:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407973:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826367.407979:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 1/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407986:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfe00/0xe21309aa81f4c5d6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c5dd expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.407999:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.408000:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.408002:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.408009:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.408015:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826367.408028:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346344448:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826367.408071:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826367.408121:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826367.408129:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346344576:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826367.408153:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826367.408189:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346344448 00000100:00100000:3.0:1787826367.408196:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346344448 00000100:00100000:3.0:1787826367.408239:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346344576 00000100:00100000:3.0:1787826367.408243:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346344576 00000100:00100000:3.0:1787826367.408288:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11219 00000100:00100000:3.0:1787826367.408298:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+13:3654:x1874663346344448:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826367.408311:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826367.408332:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c5f9 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c5f2 expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.408344:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5d:0x0].0x0 (ffff9f7967ac1f00) 00010000:00010000:3.0:1787826367.408351:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826367.408382:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346344448:12345-0@lo:103:ldlm_bl.0 Request processed in 86us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.408389:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11219 00000100:00100000:3.0:1787826367.408404:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11220 00000100:00100000:3.0:1787826367.408410:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346344576:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826367.408419:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826367.408430:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4c5dd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c5d6 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.408441:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f79785cc800) 00010000:00010000:3.0:1787826367.408447:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826367.408468:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346344576:12345-0@lo:103:ldlm_bl.0 Request processed in 58us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.408474:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11220 00000100:00100000:3.0:1787826367.408517:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346344576:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826367.408526:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346344448:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826367.414964:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.415098:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332690944 00000100:00100000:2.0:1787826367.415132:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332690944 00000100:00100000:2.0:1787826367.415198:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11221 00000100:00100000:2.0:1787826367.415210:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c83c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332690944:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:2.0:1787826367.415225:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826367.415252:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c42b lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b56 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.415281:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:2.0:1787826367.415286:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.415352:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c83c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332690944:12345-192.168.203.56@tcp:103:rm.0 Request processed in 134us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.415359:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11221 00000100:00100000:2.0:1787826367.415878:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.417437:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332691200 00000100:00100000:3.0:1787826367.417450:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332691200 00000100:00100000:3.0:1787826367.417514:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9400 00000100:00100000:3.0:1787826367.417536:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226693:x1874663332691200:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:3.0:1787826367.417765:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:226693:x1874663332691200:12345-192.168.203.56@tcp:35:rm.0 Request processed in 241us (1888us total) trans 4294985490 rc 0/0 00000100:00100000:3.0:1787826367.417773:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9400 00000100:00100000:2.0:1787826367.442149:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.442722:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332691456 00000100:00100000:0.0:1787826367.442732:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332691456 00000100:00100000:0.0:1787826367.442798:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68698 00000100:00100000:0.0:1787826367.442807:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:226693:x1874663332691456:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826367.442872:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826367.442900:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4c59e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bcd expref: 14 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.442925:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c566 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb1 expref: 13 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.442937:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c439 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b5d expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.442947:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:0.0:1787826367.444621:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826367.444686:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.444699:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.444711:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.444737:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346344704:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787826367.444804:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.444840:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826367.444873:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346344704 00000100:00100000:0.0:1787826367.444879:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346344704 00000100:00100000:0.0:1787826367.444928:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24158 00000100:00100000:0.0:1787826367.444938:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6528:x1874663346344704:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787826367.447069:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826367.447101:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c60e expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.447155:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.447168:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.447177:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.447210:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.447226:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 7 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.447241:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcb100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346344832:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826367.447384:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 7 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.447395:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 7 pid: 35098 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.447405:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc9c00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346344960:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826367.447430:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.460283:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.460325:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.460404:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcb100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346344832:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826367.460417:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc9c00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346344960:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826367.460460:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460474:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826367.460477:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826367.460481:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 11 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460491:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:2.0:1787826367.460550:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6528:x1874663346344704:12345-0@lo:101:mdt00_001.0 Request processed in 15613us (15745us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.460559:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24158 00000100:00100000:2.0:1787826367.460617:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346344704:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826367.460635:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460645:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.460670:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332691584 00000100:00100000:2.0:1787826367.460677:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332691584 00010000:00010000:2.0:1787826367.460707:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 8 pid: 35098 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.460731:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332691712 00000100:00100000:2.0:1787826367.460734:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332691712 00010000:00010000:2.0:1787826367.460756:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 8 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.460775:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22071 00000100:00100000:2.0:1787826367.460784:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2393:x1874663332691584:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826367.460797:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826367.460804:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 8 pid: 35098 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460814:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 8 pid: 35098 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460835:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4c40f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b4f expref: 8 pid: 35098 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460848:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826367.460853:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460860:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.460900:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332691584:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 117us (627us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.460911:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22071 00000100:00100000:2.0:1787826367.460925:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22072 00000100:00100000:2.0:1787826367.460932:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332691712:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826367.460939:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826367.460945:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 8 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460970:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 8 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460984:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c478 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b79 expref: 8 pid: 35095 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.460995:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826367.461000:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461009:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461019:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461053:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 10 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.461074:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346345088:0@lo:105: 00000100:00100000:2.0:1787826367.461140:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.461173:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc900 going to sleep for 11 seconds 00000100:00080000:2.0:1787826367.461187:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826367.461201:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346345088 00000100:00100000:2.0:1787826367.461204:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346345088 00000100:00100000:2.0:1787826367.461232:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7927 00000100:00100000:2.0:1787826367.461239:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346345088:12345-0@lo:105: 00010000:00010000:2.0:1787826367.461249:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461273:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461281:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461289:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461297:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461305:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461323:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43400) 00000100:00100000:2.0:1787826367.461355:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346345088:12345-0@lo:105: Request processed in 116us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.461362:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7927 00010000:00010000:2.0:1787826367.461391:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461399:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826367.461426:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787826367.461463:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461470:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461478:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.461493:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346345216:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787826367.461520:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.461538:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.461552:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346345216 00000100:00100000:2.0:1787826367.461556:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346345216 00000100:00100000:2.0:1787826367.461584:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24159 00000100:00100000:2.0:1787826367.461590:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6528:x1874663346345216:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826367.461605:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826367.461619:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c623 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c61c expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461641:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c623 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c61c expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461652:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c623 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c61c expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461659:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe500) 00010000:00010000:2.0:1787826367.461662:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826367.461664:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826367.461693:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346345216:12345-0@lo:101:mdt00_001.0 Request processed in 102us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.461699:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24159 00000100:00100000:2.0:1787826367.461726:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346345216:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826367.461737:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.461745:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826367.461757:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787826367.461790:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346345344:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826367.461815:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.461830:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.461844:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346345344 00000100:00100000:2.0:1787826367.461848:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346345344 00000100:00100000:2.0:1787826367.461871:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24160 00000100:00100000:2.0:1787826367.461878:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346345344:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826367.461940:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346345344:12345-0@lo:1000:mdt00_001.0 Request processed in 62us (124us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.461946:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24160 00000100:00100000:2.0:1787826367.462064:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d816a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346345344:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826367.462131:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346345472:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826367.462157:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.462172:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.462184:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346345472 00000100:00100000:2.0:1787826367.462187:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346345472 00000100:00100000:2.0:1787826367.462207:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24161 00000100:00100000:2.0:1787826367.462214:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346345472:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787826367.462320:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346345472:12345-0@lo:1000:mdt00_001.0 Request processed in 106us (161us total) trans 0 rc 1/1 00000100:00100000:2.0:1787826367.462327:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24161 00000100:00100000:2.0:1787826367.462352:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346345472:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787826367.462393:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.462402:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.462411:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.462422:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346345600:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787826367.462445:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.462458:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.462472:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346345600 00000100:00100000:2.0:1787826367.462475:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346345600 00000100:00100000:2.0:1787826367.462497:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24162 00000100:00100000:2.0:1787826367.462503:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6528:x1874663346345600:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826367.462510:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826367.462523:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c631 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c62a expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.462542:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.462552:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c631 lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c62a expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.462573:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.462587:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 7 pid: 35094 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.462598:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346345728:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826367.462658:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.462689:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346345088:0@lo:105: 00010000:00010000:2.0:1787826367.462702:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.462722:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332691712:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1790us (2396us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.462733:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22072 00000100:00100000:1.0:1787826367.477753:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346345856:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826367.477830:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.477899:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346345856 00000100:00100000:1.0:1787826367.477919:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346345856 00000100:00100000:1.0:1787826367.477979:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6268 00000100:00100000:1.0:1787826367.477988:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346345856:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826367.478011:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826367.478057:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346345856:12345-0@lo:13:osp-pre-0-1.0 Request processed in 71us (227us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.478065:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6268 00000100:00100000:1.0:1787826367.478135:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346345856:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826367.478405:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826367.478416:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346346112:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826367.478454:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.478610:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.478634:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332691840 00000100:00100000:2.0:1787826367.478639:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332691840 00010000:00010000:2.0:1787826367.478667:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 5/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 7 pid: 35094 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.478695:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22073 00000100:00100000:2.0:1787826367.478702:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332691840:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826367.478714:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826367.478721:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 5/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 7 pid: 35094 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478733:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 7 pid: 35094 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478755:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:2.0:1787826367.478761:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c631 lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c62a expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478772:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c631 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4c62a expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478785:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c631 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4c62a expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478812:0:8459:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c631 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c62a expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.478823:0:8459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346346240:0@lo:105: 00000100:00100000:2.0:1787826367.478852:0:8459:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.478882:0:8459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.478898:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346346240 00000100:00100000:2.0:1787826367.478902:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346346240 00000100:00100000:2.0:1787826367.478929:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7928 00000100:00100000:2.0:1787826367.478935:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346346240:12345-0@lo:105: 00010000:00010000:2.0:1787826367.478946:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478967:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478976:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478983:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.478990:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479000:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479015:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f78490ee800) 00000100:00100000:2.0:1787826367.479050:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8459:x1874663346346240:12345-0@lo:105: Request processed in 114us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.479056:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7928 00000100:00100000:2.0:1787826367.479086:0:8459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:8459:1874663346346240:0@lo:105: 00010000:00010000:2.0:1787826367.479100:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.479167:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332691840:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 465us (559us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.479174:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22073 00010000:00010000:2.0:1787826367.479276:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c494 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b80 expref: 6 pid: 35094 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.479293:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346345728:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826367.479312:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c631 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c62a expref: 13 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479320:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:2.0:1787826367.479323:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826367.479326:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826367.479360:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:6528:x1874663346345600:12345-0@lo:101:mdt00_001.0 Request processed in 16857us (16914us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.479367:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24162 00000100:00100000:2.0:1787826367.479406:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346345600:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787826367.479421:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479451:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479461:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479470:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479490:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479499:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479506:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 4/0,1 mode: --/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479527:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.479539:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 11 pid: 17813 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.479550:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6528:1874663346346368:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826367.479594:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.480133:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346345984:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826367.480175:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.480222:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346346112 00000100:00100000:1.0:1787826367.480228:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346346112 00000100:00100000:1.0:1787826367.480278:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6269 00000100:00100000:1.0:1787826367.480287:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346346112:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826367.480304:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826367.480351:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346346112:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (1896us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.480358:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6269 00000100:00100000:1.0:1787826367.480380:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826367.480389:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346346496:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826367.480419:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.480448:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346345984 00000100:00100000:1.0:1787826367.480452:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346345984 00000100:00100000:1.0:1787826367.480483:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346346496 00000100:00100000:1.0:1787826367.480486:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346346496 00000100:00100000:1.0:1787826367.480503:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24163 00000100:00100000:1.0:1787826367.480509:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346345984:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826367.480545:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:1.0:1787826367.480584:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346345984:12345-0@lo:41:osp-pre-0-1.0 Request processed in 75us (409us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.480591:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24163 00000100:00100000:1.0:1787826367.480605:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24164 00000100:00100000:1.0:1787826367.480611:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346346496:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826367.480631:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443360768 avail 1297121280 00000100:00100000:1.0:1787826367.480662:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3654:x1874663346346496:12345-0@lo:41:osp-pre-1-0.0 Request processed in 51us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.480667:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24164 00000100:00100000:1.0:1787826367.480703:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346345984:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826367.480730:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daaa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346346496:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826367.481196:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346346112:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826367.482868:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346346624:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826367.482942:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.483131:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346346624 00000100:00100000:2.0:1787826367.483140:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346346624 00000100:00100000:2.0:1787826367.483189:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6270 00000100:00100000:2.0:1787826367.483198:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346346624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826367.483221:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493312576 pending 28672 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826367.483278:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346346624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (336us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.483285:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6270 00000100:00100000:1.0:1787826367.484133:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346346624:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826367.495334:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.495532:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6528:1874663346346368:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826367.495563:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.495649:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332691968 00000100:00100000:0.0:1787826367.495657:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332691968 00010000:00010000:0.0:1787826367.495698:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 12 pid: 17813 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.495745:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22074 00000100:00100000:0.0:1787826367.495756:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332691968:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826367.495771:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826367.495778:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 12 pid: 17813 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.495790:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 12 pid: 17813 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.495813:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c574 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bb8 expref: 12 pid: 17813 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.495827:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f7967ac1300) 00010000:00010000:0.0:1787826367.495832:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.495842:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.495849:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.495875:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826367.495928:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332691968:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 173us (602us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.495935:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22074 00010000:00010000:0.0:1787826367.496292:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.496300:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.496502:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.496510:0:6528:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.496521:0:6528:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.496604:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346346752:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787826367.496658:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.496690:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483623c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826367.496708:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346346752 00000100:00100000:0.0:1787826367.496713:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346346752 00000100:00100000:0.0:1787826367.496752:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24165 00000100:00100000:0.0:1787826367.496761:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:6528:x1874663346346752:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787826367.496775:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826367.496791:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c64d lrc: 2/0,0 mode: --/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c646 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.496812:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c64d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c646 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.496823:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c64d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c646 expref: 14 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.496835:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f784b4a5900) 00010000:00010000:0.0:1787826367.496837:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826367.496840:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826367.496873:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:6528:x1874663346346752:12345-0@lo:101:mdt00_001.0 Request processed in 113us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.496879:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24165 00000100:00100000:0.0:1787826367.496913:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346346752:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787826367.496927:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.496938:0:6528:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.499912:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346346880:0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787826367.501183:0:6528:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.501235:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483623c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.501345:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346346880 00000100:00100000:2.0:1787826367.501355:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346346880 00000100:00100000:2.0:1787826367.501739:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24166 00000100:00100000:2.0:1787826367.501751:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:6528:x1874663346346880:12345-0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787826367.505365:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:6528:x1874663346346880:12345-0@lo:403:mdt00_001.0 Request processed in 3606us (4176us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.505381:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24166 00000100:00100000:3.0:1787826367.505768:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6528:1874663346346880:0@lo:403:mdt00_001.0 00000040:00080000:3.0:1787826367.506907:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1234, 952 off1126512 00000040:00080000:3.0:1787826367.507100:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1552, 1032 off1570968 00000004:00080000:3.0:1787826367.507134:0:6528:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7848334480 with version 6894 00000004:00080000:0.0:1787826367.507356:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6894 rpc_version 6894 00000020:00100000:0.0:1787826367.507392:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826367.507402:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977dc8a80 x1874663346347136/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826367.507425:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346347136:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826367.507496:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.507638:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346347136 00000100:00100000:1.0:1787826367.507648:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346347136 00000100:00100000:1.0:1787826367.507703:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24167 00000100:00100000:1.0:1787826367.507714:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8b050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346347136:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826367.508066:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8b050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346347136:12345-0@lo:1000:osp_up0-1.0 Request processed in 353us (568us total) trans 8589957491 rc 0/0 00000100:00100000:1.0:1787826367.508073:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24167 00000100:00100000:0.0:1787826367.508433:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826367.508489:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346347136:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787826367.508731:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5c:0x0]: returning size 4096 00010000:00010000:1.0:1787826367.508755:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508765:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508772:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508780:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826367.508785:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508792:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508808:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19c:0x0].0x0 (ffff9f7967ac1300) 00010000:00010000:1.0:1787826367.508811:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826367.508815:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 2/0,0 mode: --/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508822:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c63f lrc: 0/0,0 mode: --/EX res: [0x240000400:0x19c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508843:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508857:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508865:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508889:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826367.508897:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19a:0x0] 00010000:00010000:1.0:1787826367.508905:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508911:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508917:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508923:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826367.508926:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508932:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.508940:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:1.0:1787826367.508943:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826367.508946:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509016:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef400/0xe21309aa81f4c638 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509029:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509037:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509044:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509064:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787826367.509074:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826367.509079:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509086:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509092:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509130:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787826367.509134:0:6528:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826367.509139:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.509147:0:6528:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826367.509228:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226693:x1874663332691456:12345-192.168.203.56@tcp:36:rm.0 Request processed in 66420us (67086us total) trans 4294985491 rc 0/0 00000100:00100000:1.0:1787826367.509237:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68698 00010000:00010000:3.0:1787826367.509759:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826367.509767:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509776:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826367.509781:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826367.509789:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509823:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509829:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826367.509856:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509863:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42200/0xe21309aa81f4c646 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c64d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509880:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826367.509883:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509889:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826367.509893:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826367.509899:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509916:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509922:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826367.509937:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.509944:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c62a lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c631 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.510036:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826367.510041:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.510051:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826367.510055:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826367.510062:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.510079:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.510085:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826367.510091:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.510099:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43400/0xe21309aa81f4c60e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.510155:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.510157:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.510159:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.510166:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826367.510184:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346347520:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826367.510240:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826367.510287:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826367.510294:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346347392:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826367.510316:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787826367.510711:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826367.512158:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346347264:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826367.512231:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.512305:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346347520 00000100:00100000:1.0:1787826367.512313:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346347520 00000100:00100000:3.0:1787826367.512605:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346347392 00000100:00100000:3.0:1787826367.512611:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346347392 00000100:00100000:1.0:1787826367.512708:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346347264 00000100:00100000:1.0:1787826367.512713:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346347264 00000100:00100000:3.0:1787826367.512766:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11222 00000100:00100000:3.0:1787826367.512777:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+17:3655:x1874663346347520:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826367.512792:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826367.512817:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4c615 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c60e expref: 17 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.512835:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826367.512840:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826367.512886:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+16:3655:x1874663346347520:12345-0@lo:103:ldlm_bl.0 Request processed in 111us (2646us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.512892:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11222 00000100:00100000:3.0:1787826367.512911:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11223 00000100:00100000:3.0:1787826367.512918:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+15:3655:x1874663346347392:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826367.512925:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826367.512935:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c631 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c62a expref: 15 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.512947:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57ab00) 00010000:00010000:3.0:1787826367.512968:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826367.512991:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+14:3655:x1874663346347392:12345-0@lo:103:ldlm_bl.0 Request processed in 73us (2673us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.512997:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11223 00000100:00100000:3.0:1787826367.513038:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346347392:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826367.513049:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346347520:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826367.514781:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11224 00000100:00100000:1.0:1787826367.514799:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c80c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346347264:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826367.514815:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826367.514840:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4c64d lrc: 0/0,0 mode: --/EX res: [0x200000401:0x19a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c646 expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.514856:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x0 (ffff9f784b4a5900) 00010000:00010000:1.0:1787826367.514867:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826367.514949:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c80c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346347264:12345-0@lo:103:ldlm_bl.0 Request processed in 157us (2717us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.514964:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11224 00000100:00100000:2.0:1787826367.515079:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346347264:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826367.603217:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.603331:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332692096 00000100:00100000:0.0:1787826367.603342:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332692096 00000100:00100000:0.0:1787826367.604099:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68699 00000100:00100000:0.0:1787826367.604182:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226701:x1874663332692096:12345-192.168.203.56@tcp:41:stat.0 00000004:00000024:0.0:1787826367.604226:0:35094:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808692224 avail 7345647616 00000100:00100000:0.0:1787826367.604327:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226701:x1874663332692096:12345-192.168.203.56@tcp:41:stat.0 Request processed in 150us (1120us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.604335:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68699 00010000:00010000:3.0:1787826367.662330:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826367.677623:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.677977:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332692224 00000100:00100000:3.0:1787826367.677990:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332692224 00000100:00100000:3.0:1787826367.678070:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68700 00000100:00100000:3.0:1787826367.678083:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226705:x1874663332692224:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826367.678157:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826367.678181:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c654 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801be2 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.678268:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.678284:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.678296:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826367.678354:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826367.678359:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826367.678380:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.678408:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c654 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 8 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.678428:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 7 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.678438:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826367.678442:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826367.678446:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826367.679315:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226705:x1874663332692224:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 1228us (1697us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.679325:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68700 00000100:00100000:2.0:1787826367.687578:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.687938:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332692352 00000100:00100000:0.0:1787826367.687949:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332692352 00000100:00100000:0.0:1787826367.688033:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68701 00000100:00100000:0.0:1787826367.688044:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226705:x1874663332692352:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:0.0:1787826367.688064:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826367.688088:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c662 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801be9 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.688188:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.688203:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.688215:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.688227:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.688237:0:24703:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c623 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c61c expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.688245:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.688294:0:24703:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c623 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4c61c expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.688313:0:24703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346347648:0@lo:104: 00000100:00100000:0.0:1787826367.688384:0:24703:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.688434:0:24703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.688642:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346347648 00000100:00100000:1.0:1787826367.688648:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346347648 00000100:00100000:1.0:1787826367.688696:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7929 00000100:00100000:1.0:1787826367.688706:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346347648:12345-0@lo:104: 00010000:00010000:1.0:1787826367.688719:0:6510:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.688804:0:6510:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826367.688811:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:24703:x1874663346347648:12345-0@lo:104: Request processed in 108us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.688818:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7929 00010000:00010000:1.0:1787826367.688843:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826367.688849:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.688857:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826367.688862:0:30445:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826367.688871:0:30445:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.688899:0:30445:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.688906:0:30445:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826367.688927:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.688934:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ef000/0xe21309aa81f4c61c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c623 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.688967:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826367.688969:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826367.688972:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826367.688979:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826367.688990:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346347776:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826367.689022:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826367.689315:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346347776 00000100:00100000:3.0:1787826367.689321:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346347776 00010000:00010000:3.0:1787826367.689347:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c623 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4c61c expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826367.689376:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22075 00000100:00100000:3.0:1787826367.689383:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3653:x1874663346347776:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826367.689392:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826367.691528:0:24703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000_UUID:24703:1874663346347648:0@lo:104: 00010000:00010000:1.0:1787826367.691563:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787826367.695686:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957491 is committed 00010000:00010000:0.0:1787826367.695872:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c623 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4c61c expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.695893:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe500) 00010000:00010000:0.0:1787826367.695899:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.695910:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.695918:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.696093:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826367.696279:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346347776:12345-0@lo:103:ldlm_bl.0 Request processed in 6895us (7254us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.696290:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22075 00000100:00100000:2.0:1787826367.696411:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daabb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346347776:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826367.696694:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696705:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696787:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696795:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696800:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696806:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696811:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826367.696816:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696823:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696837:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826367.696840:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826367.696844:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696850:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c669 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696863:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696870:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696875:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696880:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826367.696883:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696889:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696896:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe500) 00010000:00010000:1.0:1787826367.696898:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826367.696901:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.696907:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c670 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.697076:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c662 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801be9 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.697088:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826367.697091:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826367.697248:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c662 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801be9 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.697269:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:1.0:1787826367.697334:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226705:x1874663332692352:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 9291us (9766us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826367.697342:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68701 00000100:00100000:0.0:1787826367.702705:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.702897:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332692480 00000100:00100000:3.0:1787826367.702908:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332692480 00000100:00100000:3.0:1787826367.703073:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68702 00000100:00100000:3.0:1787826367.703085:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226705:x1874663332692480:12345-192.168.203.56@tcp:101:lfs.0 00010000:00010000:3.0:1787826367.703168:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826367.703191:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c677 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bf0 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.703275:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.703289:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.703300:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826367.703349:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826367.703353:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826367.703385:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.703412:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c677 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.703427:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 8 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.703435:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826367.703438:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826367.703441:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826367.703518:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226705:x1874663332692480:12345-192.168.203.56@tcp:101:lfs.0 Request processed in 436us (823us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.703643:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68702 00000100:00100000:2.0:1787826367.712817:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.712899:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332692736 00000100:00100000:1.0:1787826367.712909:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332692736 00000100:00100000:1.0:1787826367.712998:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68703 00000100:00100000:1.0:1787826367.713009:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226705:x1874663332692736:12345-192.168.203.56@tcp:36:lfs.0 00000100:00100000:1.0:1787826367.713166:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346347904:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.713254:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.713298:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00080000:2.0:1787826367.713620:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826367.713882:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346347904 00000100:00100000:3.0:1787826367.713890:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346347904 00000100:00100000:3.0:1787826367.713938:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24168 00000100:00100000:3.0:1787826367.713947:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346347904:12345-0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826367.714213:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daabb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346347904:12345-0@lo:1000:mdt00_013.0 Request processed in 265us (956us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.714221:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24168 00000100:00100000:0.0:1787826367.714315:0:35098:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957491 gen 2 00000004:00080000:0.0:1787826367.714323:0:35098:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957489 committed transno 8589957491 00000004:00080000:0.0:1787826367.714347:0:35098:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957490 committed transno 8589957491 00000004:00080000:0.0:1787826367.714358:0:35098:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957491 committed transno 8589957491 00000100:00100000:0.0:1787826367.714384:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346347904:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.714440:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348032:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.714474:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.714503:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.714688:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346348032 00000100:00100000:2.0:1787826367.714694:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346348032 00000100:00100000:2.0:1787826367.714741:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24169 00000100:00100000:2.0:1787826367.714750:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346348032:12345-0@lo:1000:mdt00_013.0 00000100:00100000:2.0:1787826367.714854:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346348032:12345-0@lo:1000:mdt00_013.0 Request processed in 105us (378us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787826367.714862:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24169 00000100:00100000:0.0:1787826367.715963:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348032:0@lo:1000:mdt00_013.0 00010000:00010000:0.0:1787826367.716039:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.716049:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.716059:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.716077:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348160:0@lo:101:mdt00_013.0 00000100:00100000:0.0:1787826367.716157:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.716186:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.716340:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346348160 00000100:00100000:1.0:1787826367.716348:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346348160 00000100:00100000:1.0:1787826367.716410:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24170 00000100:00100000:1.0:1787826367.716419:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:35098:x1874663346348160:12345-0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826367.716431:0:6533:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.716449:0:6533:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c685 expref: -99 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.716477:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.716485:0:6533:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.716493:0:6533:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.716518:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 7 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.716661:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 7 pid: 24702 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826367.716673:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346348288:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826367.716756:0:6533:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 6 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.716766:0:6533:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 6 pid: 35093 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826367.716850:0:6533:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346348416:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826367.716879:0:6533:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.724064:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.724097:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.724165:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332692864 00000100:00100000:2.0:1787826367.724175:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332692864 00010000:00010000:2.0:1787826367.724226:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 8 pid: 24702 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.724270:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332692992 00000100:00100000:2.0:1787826367.724274:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332692992 00010000:00010000:2.0:1787826367.724288:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 7 pid: 35093 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.724309:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22076 00000100:00100000:2.0:1787826367.724318:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332692864:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826367.724332:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826367.724340:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 8 pid: 24702 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.724354:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 8 pid: 24702 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.724393:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826367.724401:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.724412:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.724465:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332692864:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 147us (413us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.724483:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22076 00000100:00100000:2.0:1787826367.724504:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22077 00000100:00100000:2.0:1787826367.724511:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332692992:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826367.724521:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826367.724526:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 7 pid: 35093 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.725030:0:6533:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4c67e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf0 expref: 7 pid: 24702 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826367.725049:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346348288:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826367.725061:0:6533:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6533:1874663346348416:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826367.725097:0:6533:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.725140:0:6533:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826367.725143:0:6533:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826367.725147:0:6533:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 9 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.725154:0:6533:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf200, rc 0) 00000100:00100000:1.0:1787826367.725215:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35098:x1874663346348160:12345-0@lo:101:mdt00_013.0 Request processed in 8797us (9058us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.725222:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24170 00000100:00100000:1.0:1787826367.725276:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348160:0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826367.725295:0:35098:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.725305:0:35098:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.725641:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 7 pid: 35093 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.725659:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4c65b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801be2 expref: 7 pid: 35093 timeout: 8279 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.725671:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826367.725676:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.725686:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.725694:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.725862:0:6511:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 8 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.725883:0:6511:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346348544:0@lo:105: 00000100:00100000:2.0:1787826367.725934:0:6511:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.725988:0:6511:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.726311:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826367.726325:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346348544 00000100:00100000:1.0:1787826367.726332:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346348544 00000100:00100000:1.0:1787826367.726928:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7930 00000100:00100000:1.0:1787826367.726942:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797137ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346348544:12345-0@lo:105: 00010000:00010000:1.0:1787826367.726965:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.726976:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.726986:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.726994:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.727003:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.727011:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.727038:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d42800) 00000100:00100000:1.0:1787826367.727100:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797137ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6511:x1874663346348544:12345-0@lo:105: Request processed in 162us (1165us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.727129:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7930 00000100:00100000:2.0:1787826367.728161:0:6511:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6511:1874663346348544:0@lo:105: 00010000:00010000:2.0:1787826367.728184:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.728235:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332692992:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 3722us (4134us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.728243:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22077 00010000:00010000:1.0:1787826367.729206:0:35098:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.729221:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826367.729253:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826367.729312:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.729320:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.729332:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826367.729355:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348672:0@lo:101:mdt00_013.0 00000100:00100000:1.0:1787826367.729412:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.729442:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.729857:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346348672 00000100:00100000:2.0:1787826367.729867:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346348672 00000100:00100000:2.0:1787826367.729926:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24171 00000100:00100000:2.0:1787826367.729936:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:35098:x1874663346348672:12345-0@lo:101:mdt00_013.0 00010000:00010000:2.0:1787826367.730020:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826367.730041:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c69a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c693 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.730066:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c69a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c693 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.730079:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c69a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c693 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.730087:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe800) 00010000:00010000:2.0:1787826367.730089:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826367.730093:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826367.730368:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346348672:12345-0@lo:101:mdt00_013.0 Request processed in 431us (954us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.730376:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24171 00000100:00100000:0.0:1787826367.732427:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348672:0@lo:101:mdt00_013.0 00010000:00010000:0.0:1787826367.732461:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.732586:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826367.732621:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787826367.732680:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcb100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348800:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.732860:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.732947:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826367.733067:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346348800 00000100:00100000:3.0:1787826367.733075:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346348800 00000100:00100000:3.0:1787826367.733376:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24172 00000100:00100000:3.0:1787826367.733388:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346348800:12345-0@lo:1000:mdt00_013.0 00000100:00100000:3.0:1787826367.733491:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346348800:12345-0@lo:1000:mdt00_013.0 Request processed in 105us (630us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.733499:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24172 00000100:00100000:0.0:1787826367.733571:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcb100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348800:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.733621:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dcb100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348928:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.733649:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.734392:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.734637:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346348928 00000100:00100000:1.0:1787826367.734645:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346348928 00000100:00100000:1.0:1787826367.734701:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24173 00000100:00100000:1.0:1787826367.734711:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346348928:12345-0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.734977:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346348928:12345-0@lo:1000:mdt00_013.0 Request processed in 267us (1327us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826367.734986:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24173 00000100:00100000:1.0:1787826367.735025:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dcb100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346348928:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.735143:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349056:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.735182:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.735200:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.735212:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346349056 00000100:00100000:1.0:1787826367.735215:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346349056 00000100:00100000:1.0:1787826367.735233:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24174 00000100:00100000:1.0:1787826367.735240:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346349056:12345-0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.735309:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346349056:12345-0@lo:1000:mdt00_013.0 Request processed in 69us (126us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787826367.735316:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24174 00000100:00100000:1.0:1787826367.735344:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349056:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.735396:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349184:0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.735418:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.735432:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb7180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.735444:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346349184 00000100:00100000:1.0:1787826367.735447:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346349184 00000100:00100000:1.0:1787826367.735464:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24175 00000100:00100000:1.0:1787826367.735469:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346349184:12345-0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.735966:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346349184:12345-0@lo:1000:mdt00_013.0 Request processed in 495us (545us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.735974:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24175 00000100:00100000:0.0:1787826367.736241:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349184:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.736313:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349312:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.736640:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.736711:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.736886:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346349312 00000100:00100000:2.0:1787826367.736893:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346349312 00000100:00100000:1.0:1787826367.736958:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24176 00000100:00100000:1.0:1787826367.736969:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346349312:12345-0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.737067:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346349312:12345-0@lo:1000:mdt00_013.0 Request processed in 99us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.737073:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24176 00000100:00100000:0.0:1787826367.737226:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349312:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.737289:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349440:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826367.737317:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.737340:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.737394:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346349440 00000100:00100000:1.0:1787826367.737399:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346349440 00000100:00100000:1.0:1787826367.737486:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24177 00000100:00100000:1.0:1787826367.737495:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346349440:12345-0@lo:1000:mdt00_013.0 00000100:00100000:1.0:1787826367.737683:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:35098:x1874663346349440:12345-0@lo:1000:mdt00_013.0 Request processed in 189us (364us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.737691:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24177 00000100:00100000:0.0:1787826367.737762:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349440:0@lo:1000:mdt00_013.0 00000040:00080000:0.0:1787826367.739225:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1235, 2168 off1128680 00000040:00080000:0.0:1787826367.739339:0:35098:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1554, 2368 off1575232 00000004:00080000:0.0:1787826367.739346:0:35098:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78483626c0 with version 6895 00000004:00080000:0.0:1787826367.739374:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6895 rpc_version 6895 00000020:00100000:0.0:1787826367.739414:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787826367.739426:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7977dc8000 x1874663346349696/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826367.739450:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346349696:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826367.740150:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.740230:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826367.740353:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346349696 00000100:00100000:1.0:1787826367.740361:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346349696 00000100:00100000:1.0:1787826367.740450:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24178 00000100:00100000:1.0:1787826367.740461:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346349696:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826367.741324:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346349696:12345-0@lo:1000:osp_up0-1.0 Request processed in 860us (1173us total) trans 8589957492 rc 0/0 00000100:00100000:1.0:1787826367.741334:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24178 00000100:00100000:2.0:1787826367.741742:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346349696:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826367.742937:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.743311:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.743324:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.743352:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.743359:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.743365:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.743417:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.743424:0:35098:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.743435:0:35098:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826367.743461:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349824:0@lo:101:mdt00_013.0 00000100:00100000:1.0:1787826367.744063:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.744197:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.744322:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346349824 00000100:00100000:2.0:1787826367.744331:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346349824 00000100:00100000:2.0:1787826367.744400:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24179 00000100:00100000:2.0:1787826367.744410:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:35098:x1874663346349824:12345-0@lo:101:mdt00_013.0 00010000:00010000:2.0:1787826367.744435:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826367.744455:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c6b6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c6af expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.744492:0:125377:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c6b6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c6af expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.744506:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c6b6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c6af expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.744513:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfe700) 00010000:00010000:2.0:1787826367.744516:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826367.744520:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee800, rc 0) 00000100:00100000:2.0:1787826367.744748:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35098:x1874663346349824:12345-0@lo:101:mdt00_013.0 Request processed in 339us (687us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.744756:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24179 00000100:00100000:1.0:1787826367.744896:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346349824:0@lo:101:mdt00_013.0 00010000:00010000:1.0:1787826367.744915:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.744927:0:35098:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826367.744968:0:35098:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000808df0c0 reply state 0000000093779667 transno 4294985492 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.744981:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.744987:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826367.744989:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826367.744993:0:35098:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826367.744998:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.745005:0:35098:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826367.745023:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19b:0x0] 00000004:00010000:1.0:1787826367.745032:0:35098:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000808df0c0 reply state 0000000093779667 transno 4294985492 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.745041:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.745046:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:1.0:1787826367.745048:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826367.745050:0:35098:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787826367.745056:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5e:0x0]: returning size 4096 00000004:00080000:1.0:1787826367.745066:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826367.745078:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.745085:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.745091:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.745135:0:35098:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787826367.745140:0:35098:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826367.745145:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.745153:0:35098:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.745364:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826367.745372:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.745380:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826367.745386:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826367.745394:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.745428:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.745435:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826367.745459:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.745466:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42800/0xe21309aa81f4c685 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c68c expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.745492:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.745494:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826367.745495:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826367.745546:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226705:x1874663332692736:12345-192.168.203.56@tcp:36:lfs.0 Request processed in 32533us (32734us total) trans 4294985492 rc 0/0 00000100:00100000:1.0:1787826367.745554:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68703 00000100:00100000:2.0:1787826367.746329:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826367.746350:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346349952:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826367.746589:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.751276:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346349952 00000100:00100000:1.0:1787826367.751295:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346349952 00000100:00100000:1.0:1787826367.751396:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11225 00000100:00100000:1.0:1787826367.751408:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3652:x1874663346349952:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826367.751424:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826367.751451:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c68c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c685 expref: 12 pid: 6533 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.751482:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826367.751488:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826367.751903:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346349952:12345-0@lo:103:ldlm_bl.0 Request processed in 495us (5429us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.751913:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11225 00000100:00100000:2.0:1787826367.752183:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346349952:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826367.757216:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.757329:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332693120 00000100:00100000:1.0:1787826367.757340:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332693120 00000100:00100000:1.0:1787826367.757405:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68704 00000100:00100000:1.0:1787826367.757416:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226694:x1874663332693120:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826367.757439:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.757460:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c6bd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bf7 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.757544:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c6c4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.757558:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c6c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.757569:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c6c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826367.757618:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826367.757622:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826367.757641:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c6c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.757667:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c6bd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf7 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.757687:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c6c4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf7 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.757694:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826367.757697:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826367.757701:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826367.757760:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226694:x1874663332693120:12345-192.168.203.56@tcp:101:bash.0 Request processed in 346us (555us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.757767:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68704 00000100:00100000:3.0:1787826367.761893:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.762133:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332693248 00000100:00100000:0.0:1787826367.762144:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332693248 00000100:00100000:0.0:1787826367.762212:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68705 00000100:00100000:0.0:1787826367.762225:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226694:x1874663332693248:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826367.762253:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826367.762287:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c6cb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801bfe expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.762368:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.762395:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.762407:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.762419:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.762429:0:35095:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c69a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c693 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.762437:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.762488:0:35095:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c69a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4c693 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.762506:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346350080:0@lo:104: 00000100:00100000:0.0:1787826367.762748:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.763232:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7852f34a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826367.763406:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346350080 00000100:00100000:3.0:1787826367.763414:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346350080 00000100:00100000:3.0:1787826367.763467:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7931 00000100:00100000:3.0:1787826367.763634:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346350080:12345-0@lo:104: 00010000:00010000:3.0:1787826367.763650:0:6509:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.763870:0:6509:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826367.763878:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:35095:x1874663346350080:12345-0@lo:104: Request processed in 249us (1130us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.763885:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7931 00010000:00010000:0.0:1787826367.764401:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826367.764410:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.764421:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:0.0:1787826367.764427:0:6514:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826367.764437:0:6514:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.764473:0:6514:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.764490:0:6514:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826367.764518:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.764534:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ece400/0xe21309aa81f4c693 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c69a expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.764557:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826367.764559:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826367.764561:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826367.764666:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346350208:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826367.764737:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.765143:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346350208 00000100:00100000:0.0:1787826367.765151:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346350208 00010000:00010000:0.0:1787826367.765186:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c69a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4c693 expref: 11 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.765213:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22078 00000100:00100000:0.0:1787826367.765222:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346350208:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826367.765235:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787826367.768814:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000_UUID:35095:1874663346350080:0@lo:104: 00010000:00010000:1.0:1787826367.768851:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826367.769687:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957492 is committed 00010000:00010000:1.0:1787826367.769929:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c69a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4c693 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.770015:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe800) 00010000:00010000:1.0:1787826367.770023:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.770032:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.770040:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.770064:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826367.770282:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3654:x1874663346350208:12345-0@lo:103:ldlm_bl.0 Request processed in 5059us (5555us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.770292:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22078 00010000:00010000:0.0:1787826367.771054:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.771065:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.771254:0:35095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-mdtlov_UUID:35095:1874663346350336:0@lo:1000:mdt00_012.0 00000100:00100000:0.0:1787826367.771326:0:35095:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.771368:0:35095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb73c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826367.771508:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346350336 00000100:00100000:3.0:1787826367.771518:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346350336 00000100:00100000:3.0:1787826367.771585:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24180 00000100:00100000:3.0:1787826367.771595:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:35095:x1874663346350336:12345-0@lo:1000:mdt00_012.0 00000100:00100000:3.0:1787826367.771682:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:35095:x1874663346350336:12345-0@lo:1000:mdt00_012.0 Request processed in 89us (357us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.771691:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24180 00000100:00100000:0.0:1787826367.772053:0:35095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef100 pname:cluuid:pid:xid:nid:opc:job mdt00_012:lustre-MDT0000-mdtlov_UUID:35095:1874663346350336:0@lo:1000:mdt00_012.0 00010000:00010000:0.0:1787826367.772154:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c6e0 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772168:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c6e0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772179:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c6e0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826367.772203:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c6e0 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772220:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772226:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772232:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772238:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772244:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826367.772247:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772264:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772277:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826367.772280:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826367.772285:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772291:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4c6d2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772305:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772312:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772318:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772323:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826367.772326:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772332:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772340:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f784abfe800) 00010000:00010000:0.0:1787826367.772343:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826367.772346:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772352:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4c6d9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772391:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4c6cb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bfe expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772406:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c6e0 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bfe expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.772414:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f794b57aa00) 00010000:00010000:0.0:1787826367.772416:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826367.772418:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826367.772477:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226694:x1874663332693248:12345-192.168.203.56@tcp:101:bash.0 Request processed in 10255us (10597us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.772488:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68705 00000100:00100000:0.0:1787826367.776609:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.776943:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346350208:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826367.777301:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332693376 00000100:00100000:1.0:1787826367.777309:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332693376 00000100:00100000:1.0:1787826367.777416:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68706 00000100:00100000:1.0:1787826367.777429:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226694:x1874663332693376:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826367.777457:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.777492:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c6e7 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c05 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.777574:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c6ee lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.777589:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c6ee lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.777602:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c6ee lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826367.777731:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5e:0x0]: returning size 4096 00000004:00200000:1.0:1787826367.777737:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826367.777792:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c6e7 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c05 expref: 10 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.777811:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c6ee lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c05 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.777820:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:1.0:1787826367.777824:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787826367.777827:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf000, rc 0) 00000100:00100000:1.0:1787826367.777900:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226694:x1874663332693376:12345-192.168.203.56@tcp:101:bash.0 Request processed in 473us (1301us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.777908:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68706 00000100:00100000:0.0:1787826367.782426:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.782634:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332693504 00000100:00100000:3.0:1787826367.782644:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332693504 00000100:00100000:3.0:1787826367.782748:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68707 00000100:00100000:3.0:1787826367.782760:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226694:x1874663332693504:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826367.782790:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826367.782813:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c6f5 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c0c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.782879:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c6fc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.782894:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c6fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.782905:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c6fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826367.782970:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19d:0x0]: returning size 4096 00000004:00200000:3.0:1787826367.782975:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826367.783017:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c6f5 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c0c expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.783034:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c6fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c0c expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.783043:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:3.0:1787826367.783046:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826367.783050:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69600, rc 0) 00000100:00100000:3.0:1787826367.783141:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b758000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226694:x1874663332693504:12345-192.168.203.56@tcp:101:bash.0 Request processed in 384us (726us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.783148:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68707 00000100:00100000:2.0:1787826367.790041:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.790321:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332693632 00000100:00100000:0.0:1787826367.790341:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332693632 00000100:00100000:0.0:1787826367.790437:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68708 00000100:00100000:0.0:1787826367.790449:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226694:x1874663332693632:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826367.790472:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826367.790504:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c703 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c13 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.790583:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.790597:0:6527:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c6b6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c6af expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.790605:0:6527:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 4/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.790637:0:6527:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c6b6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xe21309aa81f4c6af expref: 8 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826367.790653:0:6527:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346350464:0@lo:104: 00000100:00100000:0.0:1787826367.790706:0:6527:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826367.790763:0:6527:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784afb73c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826367.790936:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346350464 00000100:00100000:2.0:1787826367.790941:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346350464 00000100:00100000:2.0:1787826367.790988:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7932 00000100:00100000:2.0:1787826367.790995:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346350464:12345-0@lo:104: 00010000:00010000:2.0:1787826367.791008:0:74129:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.791075:0:74129:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826367.791081:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6527:x1874663346350464:12345-0@lo:104: Request processed in 87us (375us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.791087:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7932 00010000:00010000:2.0:1787826367.791130:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826367.791137:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.791145:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826367.791151:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826367.791159:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.791185:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.791192:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826367.791213:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 1/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.791221:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecfc00/0xe21309aa81f4c6af lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xe21309aa81f4c6b6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.791239:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826367.791241:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826367.791242:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826367.791252:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346350592:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826367.791291:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826367.791326:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346350592 00000100:00100000:2.0:1787826367.791330:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346350592 00010000:00010000:2.0:1787826367.791345:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c6b6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xe21309aa81f4c6af expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826367.791366:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22079 00000100:00100000:2.0:1787826367.791372:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3655:x1874663346350592:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826367.791381:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787826367.791928:0:6527:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6527:1874663346350464:0@lo:104: 00010000:00010000:3.0:1787826367.791968:0:6527:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.792444:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c6b6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xe21309aa81f4c6af expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.792461:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfe700) 00010000:00010000:2.0:1787826367.792466:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.792475:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.792482:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.792510:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.792576:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3655:x1874663346350592:12345-0@lo:103:ldlm_bl.0 Request processed in 1204us (1284us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.792585:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22079 00000100:00100000:2.0:1787826367.792628:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346350592:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826367.793050:0:6527:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.793063:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826367.793161:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x19b:0x0]: returning size 4096 00000004:00200000:0.0:1787826367.793167:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826367.793230:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4c703 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c13 expref: 10 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.793250:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c13 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.793267:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfe700) 00010000:00010000:0.0:1787826367.793270:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787826367.793272:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826367.793413:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226694:x1874663332693632:12345-192.168.203.56@tcp:101:bash.0 Request processed in 2965us (3381us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.793421:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68708 00000100:00100000:3.0:1787826367.798241:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.798558:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332693888 00000100:00100000:1.0:1787826367.798569:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332693888 00000100:00100000:1.0:1787826367.798626:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68709 00000100:00100000:1.0:1787826367.798638:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226694:x1874663332693888:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826367.798659:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.798662:0:35080:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787826367.798692:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4c70a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c13 expref: 9 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.798712:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfe700) 00010000:00010000:1.0:1787826367.798737:0:35080:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826367.798754:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c711 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c1a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.799039:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.799052:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.799062:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.799075:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.799082:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.799088:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826367.799667:0:35080:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x524:0x0], index 1: 0 00010000:00010000:1.0:1787826367.803559:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c726 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803585:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c726 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803597:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c726 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803638:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803650:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803656:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803663:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803668:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826367.803673:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803681:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 5/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803692:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826367.803695:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826367.803699:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.803705:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c718 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826367.804401:0:35080:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fe9f575e reply state 00000000341654bb transno 8589957493 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.804419:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.804426:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f7967ac1200) 00010000:00010000:1.0:1787826367.804429:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826367.804433:0:35080:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826367.804646:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c711 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c1a expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.804673:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c726 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c1a expref: 10 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.804682:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:1.0:1787826367.804685:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826367.804687:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecee00, rc 0) 00000100:00100000:1.0:1787826367.805299:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226694:x1874663332693888:12345-192.168.203.56@tcp:101:bash.0 Request processed in 6659us (7064us total) trans 8589957493 rc 0/0 00000100:00100000:1.0:1787826367.805309:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68709 00000100:00100000:2.0:1787826367.814700:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.815154:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332694016 00000100:00100000:3.0:1787826367.815165:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332694016 00000100:00100000:3.0:1787826367.815242:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30892 00000100:00100000:3.0:1787826367.815266:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226694:x1874663332694016:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:3.0:1787826367.815284:0:8444:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826367.815307:0:8444:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c21 expref: -99 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826367.815346:0:8444:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x524:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00000100:00080000:2.0:1787826367.815987:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1787826367.816078:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=8180s v=(0 0) 00010000:00010000:3.0:1787826367.816084:0:8444:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826367.816096:0:8444:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 9 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.816147:0:8444:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 9 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826367.816162:0:8444:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826367.816237:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226694:x1874663332694016:12345-192.168.203.56@tcp:101:bash.0 Request processed in 985us (1546us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826367.816245:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30892 00000100:00100000:2.0:1787826367.820427:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.820621:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332694272 00000100:00100000:1.0:1787826367.820633:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332694272 00000100:00100000:1.0:1787826367.820698:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9401 00000100:00100000:1.0:1787826367.820711:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967aefc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:226694:x1874663332694272:12345-192.168.203.56@tcp:35:bash.0 00000100:00100000:1.0:1787826367.821054:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967aefc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226694:x1874663332694272:12345-192.168.203.56@tcp:35:bash.0 Request processed in 345us (637us total) trans 8589957494 rc 0/0 00000100:00100000:1.0:1787826367.821063:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9401 00000100:00100000:0.0:1787826367.830891:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.831518:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332694528 00000100:00100000:1.0:1787826367.831539:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332694528 00000100:00100000:1.0:1787826367.831618:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68710 00000100:00100000:1.0:1787826367.831630:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226694:x1874663332694528:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826367.831651:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.831654:0:8752:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787826367.831689:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69600/0xe21309aa81f4c6fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c0c expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.831709:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826367.831714:0:8752:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826367.831740:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c73b lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c28 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.831894:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.831909:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.831919:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.831934:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.831942:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.832312:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826367.832498:0:8752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346350720:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787826367.832897:0:8752:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826367.832938:0:8752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826367.833223:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346350720 00000100:00100000:0.0:1787826367.833230:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346350720 00000100:00100000:0.0:1787826367.833298:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24181 00000100:00100000:0.0:1787826367.833310:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346350720:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787826367.833427:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:8752:x1874663346350720:12345-0@lo:1000:mdt00_003.0 Request processed in 119us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.833435:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24181 00000100:00100000:1.0:1787826367.834315:0:8752:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957492 gen 2 00000004:00080000:1.0:1787826367.834324:0:8752:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957492 committed transno 8589957492 00000100:00100000:1.0:1787826367.834386:0:8752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8752:1874663346350720:0@lo:1000:mdt00_003.0 00000004:00080000:1.0:1787826367.835391:0:8752:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x4d1:0x0], index 0: 0 00010000:00010000:1.0:1787826367.835906:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c750 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.835927:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c750 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.835940:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c750 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836178:0:8752:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836187:0:8752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836193:0:8752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836201:0:8752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836207:0:8752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826367.836212:0:8752:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836220:0:8752:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 5/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836232:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:1.0:1787826367.836234:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826367.836240:0:8752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836245:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfa00/0xe21309aa81f4c742 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826367.836272:0:8752:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002160965a reply state 000000009c5b42b9 transno 4294985493 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836284:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836289:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x4ec94fd6 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826367.836291:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826367.836294:0:8752:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826367.836628:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c73b lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c28 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836648:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c750 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c28 expref: 11 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.836656:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:1.0:1787826367.836659:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826367.836661:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:2.0:1787826367.838295:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226694:x1874663332694528:12345-192.168.203.56@tcp:101:bash.0 Request processed in 6658us (7405us total) trans 4294985493 rc 0/0 00000100:00100000:2.0:1787826367.838308:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68710 00000100:00100000:0.0:1787826367.847302:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826367.847538:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332694656 00000100:00100000:1.0:1787826367.847548:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332694656 00000100:00100000:1.0:1787826367.847618:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30893 00000100:00100000:1.0:1787826367.847630:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226694:x1874663332694656:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826367.847652:0:35097:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826367.847677:0:35097:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c2f expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826367.847707:0:35097:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4d1:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787826367.847743:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d1:0x0].0x0 since=8180s v=(0 0) 00010000:00010000:1.0:1787826367.847748:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d1:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826367.847759:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 8 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.847786:0:35097:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 8 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826367.847800:0:35097:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826367.847864:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226694:x1874663332694656:12345-192.168.203.56@tcp:101:bash.0 Request processed in 236us (575us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826367.847872:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30893 00000100:00100000:0.0:1787826367.852637:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.852935:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332694912 00000100:00100000:3.0:1787826367.852946:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332694912 00000100:00100000:3.0:1787826367.853227:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9402 00000100:00100000:3.0:1787826367.853238:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:226694:x1874663332694912:12345-192.168.203.56@tcp:35:bash.0 00000001:00080000:3.0:1787826367.853532:0:6531:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294985494 00000100:00100000:3.0:1787826367.853646:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+14:226694:x1874663332694912:12345-192.168.203.56@tcp:35:bash.0 Request processed in 410us (1019us total) trans 4294985495 rc 0/0 00000100:00100000:3.0:1787826367.853654:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9402 00000020:00080000:3.0:1787826367.853659:0:6531:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1787826367.891959:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826367.892071:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332695040 00000100:00100000:0.0:1787826367.892082:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332695040 00000100:00100000:0.0:1787826367.892203:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11226 00000100:00100000:0.0:1787826367.892215:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c80850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+12:15672:x1874663332695040:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826367.892231:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826367.892274:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c6c4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bf7 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.892296:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826367.892308:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c726 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c1a expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.892318:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f7967ac1b00) 00010000:00010000:0.0:1787826367.892335:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4c6e0 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801bfe expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826367.892345:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f794b57aa00) 00010000:00010000:0.0:1787826367.892352:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826367.892426:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c80850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:15672:x1874663332695040:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 212us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826367.892433:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11226 00000100:00100000:0.0:1787826367.896941:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826367.897168:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332695168 00000100:00100000:2.0:1787826367.897178:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332695168 00000100:00100000:2.0:1787826367.897252:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11227 00000100:00100000:2.0:1787826367.897273:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:15673:x1874663332695168:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826367.897290:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826367.897317:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4c543 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dc2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ba3 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.897344:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc2:0x0].0x0 (ffff9f79785cc700) 00010000:00010000:2.0:1787826367.897356:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.897428:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:15673:x1874663332695168:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 158us (498us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.897436:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11227 00000100:00100000:2.0:1787826367.903221:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826367.909146:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332695296 00000100:00100000:3.0:1787826367.909162:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332695296 00000100:00100000:2.0:1787826367.909402:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11228 00000100:00100000:2.0:1787826367.909415:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+14:15673:x1874663332695296:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826367.909434:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787826367.909462:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf400/0xe21309aa81f4c750 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c28 expref: 14 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.909484:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f7967ac1900) 00010000:00010000:2.0:1787826367.909499:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c6ee lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c05 expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.909509:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:2.0:1787826367.909517:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c4a2 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5c:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b87 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826367.909538:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5c:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:2.0:1787826367.909546:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826367.909620:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+11:15673:x1874663332695296:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 208us (6410us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826367.909632:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11228 00010000:00010000:1.0:1787826368.684502:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826369.709309:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826369.851192:0:167612:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:2.0:1787826369.851265:0:167612:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000001:00080000:2.0:1787826370.556199:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973864 is committed 00000001:00080000:2.0:1787826370.556217:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973863 is committed 00000001:00100000:2.0:1787826370.556226:0:8438:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000001:00080000:2.0:1787826370.556233:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973862 is committed 00010000:00010000:1.0:1787826370.558761:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c5eb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826370.558787:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c5eb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.558843:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c5eb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.558851:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826370.558890:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42e00/0xe21309aa81f4c5e4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c5eb expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787826370.558919:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985495 is committed 00000020:00080000:1.0:1787826370.558944:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78483623c0 refcount 2 committed 0 result 0 batchid 3449 00000020:00080000:1.0:1787826370.558948:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78483626c0 00000020:00080000:1.0:1787826370.558953:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1554 00000020:00080000:1.0:1787826370.558955:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784629d200 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784dac4600 00000020:00080000:1.0:1787826370.558959:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1235 00000001:00080000:1.0:1787826370.558973:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985491 is committed 00000020:00080000:1.0:1787826370.558978:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78483349c0 refcount 2 committed 0 result 0 batchid 3448 00000020:00080000:1.0:1787826370.558981:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f796efc3200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7848334480 00000020:00080000:1.0:1787826370.558984:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1552 00000020:00080000:1.0:1787826370.558985:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78446a1100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f794619fc00 00000020:00080000:1.0:1787826370.558988:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1234 00000004:00080000:1.0:1787826370.559001:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 408 committed 00000020:00080000:1.0:1787826370.559014:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7848362840 refcount 2 committed 0 result 0 batchid 3447 00000020:00080000:1.0:1787826370.559017:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7945398300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7848362000 00000020:00080000:1.0:1787826370.559020:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1551 00000020:00080000:1.0:1787826370.559022:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f7945398e00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784da75200 00000020:00080000:1.0:1787826370.559025:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1233 00000020:00080000:1.0:1787826370.559033:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7851ff5240 refcount 2 committed 0 result 0 batchid 3446 00000020:00080000:1.0:1787826370.559036:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abfff00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7851ff53c0 00000020:00080000:1.0:1787826370.559039:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1550 00000020:00080000:1.0:1787826370.559041:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784dac5200 00000020:00080000:1.0:1787826370.559044:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1232 00000001:00080000:1.0:1787826370.559052:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985485 is committed 00000020:00080000:1.0:1787826370.559061:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3445 batchid updated 00000100:00100000:1.0:1787826370.559076:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826370.559098:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346350848:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826370.559213:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826370.559257:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346350848 00000100:00100000:3.0:1787826370.559267:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346350848 00000020:00080000:1.0:1787826370.559304:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3449 committed batchid 3445 00000100:00100000:3.0:1787826370.559328:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11229 00000100:00100000:3.0:1787826370.559341:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3653:x1874663346350848:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787826370.559356:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826370.559383:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c5eb lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19a:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c5e4 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826370.559397:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19a:0x0].0x5a734ce9 (ffff9f7967ac1a00) 00010000:00010000:3.0:1787826370.559407:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:1.0:1787826370.559433:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f79432d3e40 with version 6896 00000020:00080000:1.0:1787826370.559448:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3442 cancel update log [0x200000401:0x3:0x0].1546: rc = 0 00000100:00100000:3.0:1787826370.559453:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346350848:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 114us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826370.559462:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11229 00010000:00010000:2.0:1787826370.559595:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 3/0,1 mode: TXN/TXN res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559605:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 2/0,0 mode: TXN/TXN res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559611:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559617:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826370.559622:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 3/0,0 mode: TXN/TXN res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559628:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 4/0,0 mode: TXN/TXN res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559639:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f7967ac1c00) 00010000:00010000:2.0:1787826370.559642:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826370.559646:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 2/0,0 mode: --/TXN res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559651:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece800/0xe21309aa81f4c6a1 lrc: 0/0,0 mode: --/TXN res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559665:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559671:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559676:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559681:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826370.559684:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559689:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559696:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f7967ac1700) 00010000:00010000:2.0:1787826370.559698:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826370.559701:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559837:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c6a8 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559855:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559862:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559868:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559873:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826370.559876:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559883:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559891:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x4ec94fd6 (ffff9f7967ac1600) 00010000:00010000:2.0:1787826370.559894:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826370.559896:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.559902:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c749 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826370.559942:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6896 rpc_version 6896 00000020:00100000:2.0:1787826370.559967:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826370.559977:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7843641f80 x1874663346351104/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826370.559996:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346351104:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826370.560038:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826370.560067:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc900 going to sleep for 11 seconds 00000100:00100000:2.0:1787826370.560259:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826370.560270:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346351232:0@lo:6:osp-syn-1-1.0 00000100:00100000:2.0:1787826370.560303:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826370.560350:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346351232 00000100:00100000:2.0:1787826370.560356:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346351232 00000100:00100000:2.0:1787826370.560402:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30894 00000100:00100000:2.0:1787826370.560410:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346351232:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:2.0:1787826370.560434:0:35352:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1230 count 1 00010000:00010000:2.0:1787826370.560461:0:35352:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826370.560683:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346351104 00000100:00100000:3.0:1787826370.560688:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346351104 00000100:00100000:3.0:1787826370.560730:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24182 00000100:00100000:3.0:1787826370.560739:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346351104:12345-0@lo:1000:osp_up0-1.0 00000020:00080000:1.0:1787826370.560784:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3442 cancel update log [0x240000400:0x3:0x0].1228: rc = 0 00000004:00080000:1.0:1787826370.560830:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78483343c0 with version 6897 00000020:00080000:1.0:1787826370.560835:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3443 cancel update log [0x200000401:0x3:0x0].1547: rc = 0 00000020:00080000:1.0:1787826370.560864:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3443 cancel update log [0x240000400:0x3:0x0].1229: rc = 0 00000004:00080000:1.0:1787826370.560887:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7848334480 with version 6898 00000100:00100000:3.0:1787826370.560892:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346351104:12345-0@lo:1000:osp_up0-1.0 Request processed in 153us (851us total) trans 8589957495 rc 0/0 00000020:00080000:1.0:1787826370.560893:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3444 cancel update log [0x200000401:0x3:0x0].1548: rc = 0 00000100:00100000:3.0:1787826370.560908:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24182 00000020:00080000:1.0:1787826370.560917:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3444 cancel update log [0x240000400:0x3:0x0].1230: rc = 0 00000004:00080000:1.0:1787826370.560944:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7848334cc0 with version 6899 00000020:00080000:1.0:1787826370.560948:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3445 cancel update log [0x200000401:0x3:0x0].1549: rc = 0 00000020:00080000:1.0:1787826370.560967:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3445 cancel update log [0x240000400:0x3:0x0].1231: rc = 0 00000020:00080000:1.0:1787826370.560974:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3445 00000100:00100000:1.0:1787826370.561010:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346350848:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826370.561684:0:35352:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 8 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.561846:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x4ce:0x0].0x0 since=9s v=(0 0) 00010000:00010000:2.0:1787826370.561852:0:35352:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826370.561858:0:35352:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.561900:0:35352:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 8 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.561916:0:35352:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 8 pid: 8445 timeout: 8282 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826370.561931:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346351360:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826370.562019:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc240 going to sleep for 11 seconds 00000100:00080000:2.0:1787826370.562035:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826370.562074:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346351104:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787826370.562085:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6897 rpc_version 6897 00000020:00100000:2.0:1787826370.563157:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826370.563170:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b19c000 x1874663346351616/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826370.563192:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346351616:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826370.563257:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826370.563289:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78482bc900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826370.563404:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346351616 00000100:00100000:1.0:1787826370.563412:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346351616 00000100:00100000:2.0:1787826370.563513:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24183 00000100:00100000:2.0:1787826370.563524:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346351616:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826370.563693:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346351616:12345-0@lo:1000:osp_up0-1.0 Request processed in 171us (435us total) trans 8589957496 rc 0/0 00000100:00100000:2.0:1787826370.563701:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24183 00000100:00100000:3.0:1787826370.564081:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346351616:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826370.564120:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6898 rpc_version 6898 00000020:00100000:3.0:1787826370.564139:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826370.564144:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5c8380 x1874663346351872/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826370.564165:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346351872:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826370.564197:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826370.564221:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826370.564418:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346351872 00000100:00100000:2.0:1787826370.564423:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346351872 00000100:00100000:2.0:1787826370.564462:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24184 00000100:00100000:2.0:1787826370.564470:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346351872:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826370.564569:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346351872:12345-0@lo:1000:osp_up0-1.0 Request processed in 100us (370us total) trans 8589957497 rc 0/0 00000100:00100000:2.0:1787826370.564576:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24184 00000100:00100000:3.0:1787826370.564667:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346351872:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787826370.564675:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6899 rpc_version 6899 00000020:00100000:3.0:1787826370.564686:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826370.564690:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5cb100 x1874663346352128/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826370.564701:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346352128:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826370.564735:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826370.564750:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826370.564762:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346352128 00000100:00100000:3.0:1787826370.564766:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346352128 00000100:00100000:3.0:1787826370.572177:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24185 00000100:00100000:3.0:1787826370.572200:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346352128:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826370.572453:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346352128:12345-0@lo:1000:osp_up0-1.0 Request processed in 261us (7715us total) trans 8589957498 rc 0/0 00000100:00100000:3.0:1787826370.572461:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24185 00000100:00100000:3.0:1787826370.572532:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346352128:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826370.572999:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826370.573032:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332695424 00000100:00100000:1.0:1787826370.573038:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332695424 00000100:00080000:2.0:1787826370.573083:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787826370.573089:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 9 pid: 8445 timeout: 8282 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826370.573145:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22080 00000100:00100000:1.0:1787826370.573155:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332695424:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826370.573169:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826370.573175:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 9 pid: 8445 timeout: 8282 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.573188:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 9 pid: 8445 timeout: 8282 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.573217:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826370.573222:0:125406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4ce:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787826370.573228:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.573249:0:125406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.573270:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826370.573325:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2394:x1874663332695424:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 171us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826370.573332:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22080 00010000:00010000:3.0:1787826370.574195:0:35352:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000392ed720/0xe21309aa81f4c3fa lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801b48 expref: 8 pid: 8445 timeout: 8282 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826370.574213:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346351360:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826370.574247:0:35352:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826370.574258:0:35352:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826370.574264:0:35352:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826370.574274:0:35352:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826370.574282:0:35352:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826370.574313:0:35352:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826370.574524:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+6:3652:x1874663346351232:12345-0@lo:6:osp-syn-1-1.0 Request processed in 14113us (14218us total) trans 4294976173 rc 0/0 00000100:00100000:3.0:1787826370.574532:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30894 00000004:00080000:2.0:1787826370.575037:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784b19dc00/2, rc 0, transno 4294976173 00000100:00100000:2.0:1787826370.575044:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346351232:0@lo:6:osp-syn-1-1.0 00010000:00010000:1.0:1787826370.575144:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826370.575152:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.575168:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:1.0:1787826370.575173:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.575182:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.575200:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.575206:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f590973/0xe21309aa81f4c765 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4ce:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.575227:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826370.575229:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826370.575231:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826370.732668:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826370.788508:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826370.788545:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826370.788568:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826370.788586:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826370.788604:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826370.788620:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826370.788639:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826370.788664:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826370.788692:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332695552 00000100:00100000:2.0:1787826370.788702:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332695552 00000100:00100000:2.0:1787826370.788754:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2244 00000100:00100000:2.0:1787826370.788764:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332695552:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826370.788831:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332695552:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 68us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826370.788838:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2244 00000100:00100000:2.0:1787826370.788873:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332695808 00000100:00100000:2.0:1787826370.788877:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332695808 00000100:00100000:2.0:1787826370.788916:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332696320 00000100:00100000:2.0:1787826370.788919:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332696320 00000100:00100000:2.0:1787826370.788940:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332695680 00000100:00100000:2.0:1787826370.788942:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332695680 00000100:00100000:2.0:1787826370.788961:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332696192 00000100:00100000:2.0:1787826370.788963:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332696192 00000100:00100000:2.0:1787826370.788985:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14392 00000100:00100000:2.0:1787826370.788992:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332695808:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826370.789043:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332695808:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 51us (497us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826370.789048:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14392 00000100:00100000:2.0:1787826370.789061:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14393 00000100:00100000:2.0:1787826370.789066:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332696320:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826370.789087:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332696320:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (501us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826370.789093:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14393 00000100:00100000:2.0:1787826370.789129:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14394 00000100:00100000:2.0:1787826370.789136:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332695680:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826370.789161:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332695680:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (555us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826370.789166:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14394 00000100:00100000:2.0:1787826370.789178:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14395 00000100:00100000:2.0:1787826370.789184:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332696192:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826370.789208:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b619c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:2396:x1874663332696192:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (569us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826370.789213:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14395 00000100:00100000:2.0:1787826370.789246:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332696064 00000100:00100000:2.0:1787826370.789249:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332696064 00000100:00100000:2.0:1787826370.789278:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332695936 00000100:00100000:2.0:1787826370.789281:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332695936 00000100:00100000:2.0:1787826370.789302:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332696448 00000100:00100000:2.0:1787826370.789304:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332696448 00000100:00100000:2.0:1787826370.789323:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8892 00000100:00100000:2.0:1787826370.789329:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332696064:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826370.789353:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332696064:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (785us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826370.789359:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8892 00000100:00100000:2.0:1787826370.789370:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8893 00000100:00100000:2.0:1787826370.789376:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332695936:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826370.789396:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332695936:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (775us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826370.789406:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8893 00000100:00100000:2.0:1787826370.789416:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8894 00000100:00100000:2.0:1787826370.789422:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332696448:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826370.789442:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332696448:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (779us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826370.789447:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8894 00000100:00080000:2.0:1787826370.789481:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826370.789484:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826370.789487:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826370.789489:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826371.567746:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346352512:0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.567753:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346352640:0@lo:400:kworker.0 00000100:00100000:0.0:1787826371.567816:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346352384:0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.567816:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346352256:0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.568029:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826371.568069:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346353152:0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.568092:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826371.568135:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346353664:0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.568160:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826371.568201:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346352640 00000100:00100000:1.0:1787826371.568208:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346352640 00000100:00100000:1.0:1787826371.568277:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346353152 00000100:00100000:1.0:1787826371.568281:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346353152 00000100:00100000:1.0:1787826371.568308:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14396 00000100:00100000:1.0:1787826371.568320:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346352640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826371.568333:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826371.568363:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346353024:0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.568385:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346352640:12345-0@lo:400:kworker.0 Request processed in 67us (354us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.568398:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14396 00000100:00100000:2.0:1787826371.568400:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826371.568419:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346352512 00000100:00100000:1.0:1787826371.568423:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346352512 00000100:00100000:2.0:1787826371.568428:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346353536:0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.568444:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346353024 00000100:00100000:1.0:1787826371.568448:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346353024 00000100:00100000:2.0:1787826371.568460:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826371.568480:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14397 00000100:00100000:1.0:1787826371.568489:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346353152:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826371.568508:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14398 00000100:00100000:2.0:1787826371.568518:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346352512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.568538:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346353152:12345-0@lo:400:kworker.0 Request processed in 49us (443us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.568544:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14397 00000100:00100000:1.0:1787826371.568560:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14399 00000100:00100000:1.0:1787826371.568568:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346353024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826371.568572:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346352512:12345-0@lo:400:kworker.0 Request processed in 55us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.568601:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346353024:12345-0@lo:400:kworker.0 Request processed in 34us (205us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.568608:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14399 00000100:00100000:1.0:1787826371.568628:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346353664 00000100:00100000:1.0:1787826371.568632:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346353664 00000100:00100000:1.0:1787826371.568660:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346353536 00000100:00100000:1.0:1787826371.568663:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346353536 00000100:00100000:1.0:1787826371.568680:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8895 00000100:00100000:1.0:1787826371.568686:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346353664:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.568716:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346353664:12345-0@lo:400:kworker.0 Request processed in 30us (555us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.568724:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8895 00000100:00100000:2.0:1787826371.568734:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14398 00000100:00100000:1.0:1787826371.568740:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8896 00000100:00100000:1.0:1787826371.568749:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f684a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346353536:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.568781:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f684a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346353536:12345-0@lo:400:kworker.0 Request processed in 33us (321us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.568787:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8896 00000100:00100000:2.0:1787826371.568805:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346352512:0@lo:400:kworker.0 00000100:00100000:2.0:1787826371.568819:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346353024:0@lo:400:kworker.0 00000100:00100000:2.0:1787826371.568829:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973864 gen 2 00000004:00080000:2.0:1787826371.568844:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784d816d80, transno 4294973864 00000100:00100000:2.0:1787826371.568856:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346353536:0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.569552:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826371.569599:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346352768:0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.569628:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826371.569649:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346353280:0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.569675:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826371.569683:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346352256 00000100:00100000:2.0:1787826371.569691:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346352256 00000100:00100000:3.0:1787826371.569695:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346353792:0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.569725:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826371.569734:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2245 00000100:00100000:2.0:1787826371.569745:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784331e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346352256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.569767:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346352768 00000100:00100000:3.0:1787826371.569773:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346352768 00000100:00100000:2.0:1787826371.569797:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784331e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346352256:12345-0@lo:400:kworker.0 Request processed in 52us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826371.569805:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2245 00000100:00100000:3.0:1787826371.569824:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346353792 00000100:00100000:3.0:1787826371.569829:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346353792 00000100:00100000:3.0:1787826371.569853:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14400 00000100:00100000:3.0:1787826371.569860:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346352768:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.569894:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346352768:12345-0@lo:400:kworker.0 Request processed in 34us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826371.569901:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14400 00000100:00100000:3.0:1787826371.569918:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14401 00000100:00100000:3.0:1787826371.569925:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346353792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.569951:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346353792:12345-0@lo:400:kworker.0 Request processed in 27us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826371.569956:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14401 00000100:00100000:3.0:1787826371.569975:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346353280 00000100:00100000:3.0:1787826371.569979:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346353280 00000100:00100000:3.0:1787826371.570012:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8897 00000100:00100000:3.0:1787826371.570018:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346353280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.570047:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+6:3654:x1874663346353280:12345-0@lo:400:kworker.0 Request processed in 30us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826371.570053:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8897 00000100:00100000:3.0:1787826371.570254:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc32300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346352256:0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.570267:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346352768:0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.570277:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346353280:0@lo:400:kworker.0 00000100:00100000:3.0:1787826371.570287:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346353792:0@lo:400:kworker.0 00000100:00080000:1.0:1787826371.570496:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826371.570502:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826371.570505:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826371.570507:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826371.570542:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346352640:0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.570554:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346353152:0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.570566:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346353664:0@lo:400:kworker.0 00000100:00100000:0.0:1787826371.570623:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826371.570656:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346352896:0@lo:400:kworker.0 00000100:00100000:0.0:1787826371.570683:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826371.570698:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346353408:0@lo:400:kworker.0 00000100:00100000:0.0:1787826371.570718:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826371.570770:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346352384 00000100:00100000:1.0:1787826371.570775:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346352384 00000100:00100000:1.0:1787826371.570881:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346353408 00000100:00100000:1.0:1787826371.570885:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346353408 00000100:00100000:1.0:1787826371.570903:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24186 00000100:00100000:1.0:1787826371.570910:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346352384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.570957:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:3655:x1874663346352384:12345-0@lo:400:kworker.0 Request processed in 47us (334us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.570963:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24186 00000100:00100000:1.0:1787826371.570976:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24187 00000100:00100000:1.0:1787826371.570981:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346353408:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826371.571007:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346353408:12345-0@lo:400:kworker.0 Request processed in 27us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.571013:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24187 00000100:00100000:2.0:1787826371.571163:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346352896 00000100:00100000:2.0:1787826371.571168:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346352896 00000100:00100000:2.0:1787826371.571206:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8898 00000100:00100000:2.0:1787826371.571212:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346352896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826371.571257:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346352896:12345-0@lo:400:kworker.0 Request processed in 46us (573us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826371.571263:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8898 00000100:00080000:2.0:1787826371.571286:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826371.573969:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346352384:0@lo:400:kworker.0 00000100:00100000:0.0:1787826371.573988:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973862 gen 2 00000004:00080000:0.0:1787826371.573994:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784a11ea00, transno 4294973862 00000100:00100000:0.0:1787826371.574019:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346352896:0@lo:400:kworker.0 00000100:00100000:0.0:1787826371.574032:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985485 gen 2 00000004:00080000:0.0:1787826371.574037:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985484 committed transno 4294985485 00000020:00080000:0.0:1787826371.574043:0:3655:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f78483346c0 refcount 2 committed 0 result 0 batchid 4294979090 00000020:00080000:0.0:1787826371.574048:0:3655:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7848334c00 00000020:00080000:0.0:1787826371.574051:0:3655:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].947 00000020:00080000:0.0:1787826371.574054:0:3655:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f7851815400 00000020:00080000:0.0:1787826371.574057:0:3655:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].944 00000004:00080000:0.0:1787826371.574093:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985485 committed transno 4294985485 00000100:00100000:0.0:1787826371.574131:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346353408:0@lo:400:kworker.0 00000020:00080000:1.0:1787826371.574242:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979090 committed batchid 4294979089 00000020:00080000:1.0:1787826371.574311:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979089 00010000:00010000:1.0:1787826371.756680:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826371.814452:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826371.814559:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826371.814692:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332696704 00000100:00100000:3.0:1787826371.814702:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332696704 00000100:00100000:3.0:1787826371.814767:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30895 00000100:00100000:3.0:1787826371.814777:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332696704:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:3.0:1787826371.814908:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332696704:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 131us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826371.814916:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30895 00000100:00100000:0.0:1787826371.815137:0:20174:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332696576 00000100:00100000:0.0:1787826371.815142:0:20174:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332696576 00002000:00100000:0.0:1787826371.815175:0:20174:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x524:0x0] (0->5) req@ffff9f784f686a00 x1874663332696576/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:727/0 lens 496/0 e 0 to 0 dl 1787826382 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826371.815186:0:20174:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f686a00 with x1874663332696576 ext(0->5) 00002000:00010000:0.0:1787826371.815193:0:20174:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784f686a00 00000100:00100000:0.0:1787826371.815219:0:20174:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27669 00000100:00100000:0.0:1787826371.815236:0:20174:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2396:x1874663332696576:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:0.0:1787826371.816098:0:20174:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:2396:x1874663332696576:12345-192.168.203.56@tcp:4:bash.0 Request processed in 867us (1653us total) trans 4294976174 rc 0/0 00000100:00100000:0.0:1787826371.816148:0:20174:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27669 00002000:00100000:0.0:1787826371.816157:0:20174:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x524:0x0] (0->5) req@ffff9f784f686a00 x1874663332696576/t4294976174(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:727/0 lens 496/448 e 0 to 0 dl 1787826382 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826371.816166:0:20174:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f784f686a00 with x1874663332696576 ext(0->5) 00002000:00010000:0.0:1787826371.816170:0:20174:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f784f686a00 00000001:00080000:0.0:1787826371.818697:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976174 is committed 00000001:00080000:0.0:1787826371.818720:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976173 is committed 00000100:00100000:2.0:1787826371.824613:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826371.824671:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826371.824855:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332696960 00000100:00100000:1.0:1787826371.824864:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332696960 00000100:00100000:1.0:1787826371.824932:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30896 00000100:00100000:1.0:1787826371.824943:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332696960:12345-192.168.203.56@tcp:16:kworker.0 00000100:00100000:1.0:1787826371.825058:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332696960:12345-192.168.203.56@tcp:16:kworker.0 Request processed in 116us (385us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826371.825065:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30896 00000100:00100000:0.0:1787826371.827069:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332696832 00000100:00100000:0.0:1787826371.827084:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332696832 00002000:00100000:0.0:1787826371.827184:0:8448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4d1:0x0] (0->5) req@ffff9f7850ad5880 x1874663332696832/t0(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:727/0 lens 496/0 e 0 to 0 dl 1787826382 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826371.827196:0:8448:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7850ad5880 with x1874663332696832 ext(0->5) 00002000:00010000:0.0:1787826371.827203:0:8448:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7850ad5880 00000100:00100000:0.0:1787826371.827251:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27670 00000100:00100000:0.0:1787826371.827262:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332696832:12345-192.168.203.56@tcp:4:bash.0 00000100:00100000:0.0:1787826371.831178:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332696832:12345-192.168.203.56@tcp:4:bash.0 Request processed in 3905us (6562us total) trans 4294973865 rc 0/0 00000100:00100000:0.0:1787826371.831192:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27670 00002000:00100000:0.0:1787826371.831206:0:8448:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4d1:0x0] (0->5) req@ffff9f7850ad5880 x1874663332696832/t4294973865(0) o4->fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp:727/0 lens 496/448 e 0 to 0 dl 1787826382 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787826371.831216:0:8448:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f7850ad5880 with x1874663332696832 ext(0->5) 00002000:00010000:0.0:1787826371.831223:0:8448:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9f7850ad5880 00000100:00100000:2.0:1787826372.076735:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346353920:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826372.076823:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826372.077016:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346353920 00000100:00100000:1.0:1787826372.077024:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346353920 00000100:00100000:1.0:1787826372.077092:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6271 00000100:00100000:1.0:1787826372.077125:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346353920:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826372.077147:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683135488 00000100:00100000:1.0:1787826372.077195:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346353920:12345-0@lo:13:osp-pre-1-1.0 Request processed in 93us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826372.077202:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6271 00000100:00080000:2.0:1787826372.077384:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826372.077419:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294976173 gen 2 00000004:00080000:2.0:1787826372.077426:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784b19dc00, transno 4294976173 00000100:00100000:2.0:1787826372.077453:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346353920:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826372.588425:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346354048:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826372.588509:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826372.588561:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826372.588568:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b752680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346354176:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826372.588604:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826372.588655:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346354304:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826372.588763:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826372.588825:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346354048 00000100:00100000:1.0:1787826372.588834:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346354048 00000100:00100000:1.0:1787826372.588892:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346354176 00000100:00100000:1.0:1787826372.588897:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346354176 00000100:00100000:1.0:1787826372.588937:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24188 00000100:00100000:1.0:1787826372.588949:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346354048:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826372.588986:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443360768 avail 1297121280 00000100:00100000:1.0:1787826372.589041:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346354048:12345-0@lo:41:osp-pre-1-0.0 Request processed in 95us (531us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826372.589050:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24188 00000100:00100000:1.0:1787826372.589085:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24189 00000100:00100000:1.0:1787826372.589094:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346354176:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826372.589135:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442742272 avail 1296502784 00000100:00100000:1.0:1787826372.589177:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874663346354176:12345-0@lo:41:osp-pre-0-1.0 Request processed in 84us (572us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826372.589185:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24189 00000100:00100000:1.0:1787826372.589216:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346354304 00000100:00100000:1.0:1787826372.589220:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346354304 00000100:00100000:1.0:1787826372.589343:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6272 00000100:00100000:1.0:1787826372.589353:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346354304:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826372.589370:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826372.589412:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346354304:12345-0@lo:13:osp-pre-0-1.0 Request processed in 62us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826372.589419:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6272 00000100:00100000:1.0:1787826372.589448:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826372.589458:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346354560:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826372.589488:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826372.589534:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346354560 00000100:00100000:1.0:1787826372.589535:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346354432:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826372.589541:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346354560 00000100:00100000:1.0:1787826372.589576:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826372.589584:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6273 00000100:00100000:2.0:1787826372.589596:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346354560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826372.589610:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493226560 pending 28672 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826372.589641:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346354304:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826372.589661:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346354560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (169us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826372.589668:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6273 00000100:00100000:2.0:1787826372.589682:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346354432 00000100:00100000:2.0:1787826372.589686:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346354432 00000100:00100000:2.0:1787826372.589702:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6274 00000100:00100000:2.0:1787826372.589707:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346354432:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826372.589717:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683135488 00000100:00100000:2.0:1787826372.593125:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346354432:12345-0@lo:13:osp-pre-1-0.0 Request processed in 3388us (3522us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826372.593142:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6274 00000100:00100000:3.0:1787826372.593286:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adfa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346354432:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826372.594066:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784cc33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346354560:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826372.596179:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346354176:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826372.596206:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346354048:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1787826372.780374:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826373.109165:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826373.109247:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332697088 00000100:00100000:1.0:1787826373.109260:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332697088 00000100:00100000:1.0:1787826373.109388:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68711 00000100:00100000:1.0:1787826373.109402:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226875:x1874663332697088:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:1.0:1787826373.109423:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826373.109447:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c76c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c36 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.109516:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.109530:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.109544:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826373.109603:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826373.109607:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826373.109628:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.109658:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf200/0xe21309aa81f4c76c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.109674:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.109683:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826373.109686:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826373.109690:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece400, rc 0) 00000100:00100000:1.0:1787826373.109755:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226875:x1874663332697088:12345-192.168.203.56@tcp:101:cat.0 Request processed in 357us (601us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.109762:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68711 00000100:00080000:2.0:1787826373.109830:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826373.116985:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826373.118043:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332697216 00000100:00100000:0.0:1787826373.118055:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332697216 00000100:00100000:0.0:1787826373.118153:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68712 00000100:00100000:0.0:1787826373.118164:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226875:x1874663332697216:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:0.0:1787826373.118185:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826373.118208:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c77a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c3d expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118289:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118303:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118314:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118329:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118335:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118341:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118428:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c78f lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118442:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c78f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118449:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c78f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826373.118473:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c78f lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118488:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118494:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118500:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118507:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118512:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826373.118517:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118524:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118536:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826373.118539:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826373.118544:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118550:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42200/0xe21309aa81f4c781 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118563:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118569:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118575:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118580:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826373.118583:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118589:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118595:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f794b57a700) 00010000:00010000:0.0:1787826373.118598:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826373.118601:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118607:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4c788 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118649:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4c77a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c3d expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118667:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c78f lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c3d expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.118674:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:0.0:1787826373.118677:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826373.118679:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826373.118783:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226875:x1874663332697216:12345-192.168.203.56@tcp:101:cat.0 Request processed in 621us (1808us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826373.118791:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68712 00000100:00100000:3.0:1787826373.121416:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826373.122310:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332697344 00000100:00100000:2.0:1787826373.122322:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332697344 00000100:00100000:2.0:1787826373.122421:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68713 00000100:00100000:2.0:1787826373.122660:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:226875:x1874663332697344:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826373.122685:0:8752:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826373.122713:0:8752:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c796 lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c44 expref: -99 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.122782:0:8752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c79d lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.122794:0:8752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c79d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.122807:0:8752:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c79d lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8752 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826373.122914:0:8752:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5e:0x0]: returning size 4096 00000004:00200000:2.0:1787826373.122919:0:8752:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826373.122959:0:8752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c796 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c44 expref: 7 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.122980:0:8752:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c79d lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c44 expref: 6 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.122991:0:8752:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:2.0:1787826373.122994:0:8752:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1787826373.122997:0:8752:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efe00, rc 0) 00000100:00100000:2.0:1787826373.123078:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226875:x1874663332697344:12345-192.168.203.56@tcp:101:cat.0 Request processed in 647us (1673us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.123086:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68713 00000100:00080000:3.0:1787826373.123262:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826373.130264:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826373.130322:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332697472 00000100:00100000:3.0:1787826373.130333:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332697472 00000100:00100000:3.0:1787826373.130416:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68714 00000100:00100000:3.0:1787826373.130427:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226875:x1874663332697472:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826373.130448:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826373.130476:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c7a4 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c4b expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.130544:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.130557:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.130569:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826373.130629:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19d:0x0]: returning size 4096 00000004:00200000:3.0:1787826373.130635:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787826373.130674:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c7a4 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.130692:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.130700:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826373.130704:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826373.130707:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826373.130784:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226875:x1874663332697472:12345-192.168.203.56@tcp:101:cat.0 Request processed in 359us (529us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826373.130791:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68714 00000100:00100000:3.0:1787826373.134465:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826373.135166:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332697600 00000100:00100000:2.0:1787826373.135177:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332697600 00000100:00100000:2.0:1787826373.135248:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68715 00000100:00100000:2.0:1787826373.135259:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226875:x1874663332697600:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826373.135282:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826373.135307:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c7b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c52 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.135372:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.135386:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.135396:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826373.135449:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x19b:0x0]: returning size 4096 00000004:00200000:2.0:1787826373.135454:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787826373.135495:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4c7b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.135512:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.135521:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826373.135524:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787826373.135528:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee400, rc 0) 00000100:00100000:2.0:1787826373.135606:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226875:x1874663332697600:12345-192.168.203.56@tcp:101:cat.0 Request processed in 349us (1149us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.135613:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68715 00000100:00100000:2.0:1787826373.151270:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826373.151590:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332697856 00000100:00100000:3.0:1787826373.151602:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332697856 00000100:00100000:3.0:1787826373.151665:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68716 00000100:00100000:3.0:1787826373.151676:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226875:x1874663332697856:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826373.151697:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826373.151729:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c7c0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c59 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.151810:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.151824:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.151836:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.151847:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.151854:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.151860:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00020000:3.0:1787826373.151953:0:11629:0:(mdt_open.c:1594:mdt_reint_open()) cfs_race id 16a sleeping 00000020:00080000:3.0:1787826373.626399:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979090 batchid updated 00000001:00080000:3.0:1787826373.626438:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957498 is committed 00000020:00080000:0.0:1787826373.626461:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979090 00000001:00080000:3.0:1787826373.626465:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957494 is committed 00010000:00010000:3.0:1787826373.626499:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.626513:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.626522:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.626530:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826373.626536:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826373.626542:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7848d5ea80 with version 1842 00010000:00010000:3.0:1787826373.626544:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787826373.626557:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979090 cancel update log [0x240000404:0x2:0x0].947: rc = 0 00010000:00010000:3.0:1787826373.626566:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f7967ac1200) 00010000:00010000:3.0:1787826373.626570:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826373.626575:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.626581:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c71f lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826373.626607:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1842 rpc_version 1842 00000020:00100000:3.0:1787826373.626647:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000020:00080000:0.0:1787826373.626656:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979090 cancel update log [0x200000bd0:0x2:0x0].944: rc = 0 00000100:00100000:3.0:1787826373.626659:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7851886680 x1874663346354816/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000020:00080000:0.0:1787826373.626667:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979090 00000100:00100000:3.0:1787826373.626689:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851886680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346354816:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826373.626784:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.627381:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826373.627968:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332697984 00000100:00100000:2.0:1787826373.627978:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332697984 00000100:00100000:2.0:1787826373.628210:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68717 00000100:00100000:3.0:1787826373.628544:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318540 going to sleep for 11 seconds 00000100:00080000:3.0:1787826373.628574:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826373.628827:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226877:x1874663332697984:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826373.628849:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826373.628872:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c7d5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c60 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.629013:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.629028:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.629040:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826373.629094:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826373.629098:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826373.629141:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.629178:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4c7d5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.629197:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.629205:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826373.629208:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826373.629211:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826373.629285:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226877:x1874663332697984:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 469us (1907us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.629293:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68717 00000100:00100000:0.0:1787826373.630051:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346354816 00000100:00100000:0.0:1787826373.630061:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346354816 00000100:00100000:0.0:1787826373.630138:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24190 00000100:00100000:0.0:1787826373.630148:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:30909:x1874663346354816:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787826373.630918:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346354816:12345-0@lo:1000:osp_up1-0.0 Request processed in 770us (4132us total) trans 4294985496 rc 0/0 00000100:00100000:0.0:1787826373.630927:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24190 00000100:00100000:2.0:1787826373.631062:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851886680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346354816:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826373.635226:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826373.635370:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332698112 00000100:00100000:3.0:1787826373.635381:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332698112 00000100:00100000:3.0:1787826373.635443:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68718 00000100:00100000:3.0:1787826373.635455:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226877:x1874663332698112:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:3.0:1787826373.635477:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826373.635510:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c7e3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c67 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635586:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635599:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635611:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635625:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635632:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635639:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635717:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635727:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635733:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826373.635757:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635771:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635777:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635782:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635792:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635797:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826373.635802:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635809:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.635822:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826373.635825:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826373.636188:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636197:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4c7ea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636290:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636296:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636302:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636308:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826373.636311:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636321:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636330:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f79460a2b00) 00010000:00010000:3.0:1787826373.636333:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826373.636336:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636342:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c7f1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636380:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c7e3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636395:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.636401:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:3.0:1787826373.636403:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826373.636406:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69000, rc 0) 00000100:00100000:3.0:1787826373.641573:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226877:x1874663332698112:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 6105us (6349us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826373.641588:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68718 00000100:00100000:3.0:1787826373.663185:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826373.663267:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332698240 00000100:00100000:0.0:1787826373.663277:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332698240 00000100:00100000:0.0:1787826373.663357:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68719 00000100:00100000:0.0:1787826373.663368:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:226877:x1874663332698240:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826373.663391:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826373.663414:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4c7ff lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c6e expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.663600:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4c806 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.663615:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4c806 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.663626:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4c806 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826373.663732:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5e:0x0]: returning size 4096 00000004:00200000:0.0:1787826373.663737:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787826373.663780:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43400/0xe21309aa81f4c7ff lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c6e expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.663799:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4c806 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c6e expref: 6 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.663807:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:0.0:1787826373.663810:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787826373.663814:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42200, rc 0) 00000100:00080000:3.0:1787826373.663946:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826373.664148:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226877:x1874663332698240:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 780us (973us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826373.664156:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68719 00000100:00100000:2.0:1787826373.683041:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826373.683148:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332698368 00000100:00100000:0.0:1787826373.683158:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332698368 00000100:00100000:0.0:1787826373.683226:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68720 00000100:00100000:0.0:1787826373.683238:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:226877:x1874663332698368:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:0.0:1787826373.683260:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826373.683283:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4c80d lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c75 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683357:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683372:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683383:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683398:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683406:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683413:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683584:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c822 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683593:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c822 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683599:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c822 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826373.683648:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5f:0x0]: returning size 6 00000004:00200000:0.0:1787826373.683652:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826373.683672:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c822 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683685:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c822 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683695:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683700:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683706:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683712:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683717:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826373.683721:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683728:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683740:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:0.0:1787826373.683743:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826373.683747:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683756:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43600/0xe21309aa81f4c814 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683770:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683775:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683780:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683785:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826373.683787:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683792:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683799:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x4ec94fd6 (ffff9f794b57a700) 00010000:00010000:0.0:1787826373.683801:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826373.683804:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.683810:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c81b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.684327:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d43200/0xe21309aa81f4c80d lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c75 expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.684348:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c822 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c75 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.684357:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:0.0:1787826373.684359:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826373.684362:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826373.684434:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:226877:x1874663332698368:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 1197us (1404us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826373.684443:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68720 00000100:00100000:0.0:1787826373.696239:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826373.696345:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332698496 00000100:00100000:2.0:1787826373.696356:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332698496 00000100:00100000:2.0:1787826373.696418:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68721 00000100:00100000:2.0:1787826373.696430:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:226877:x1874663332698496:12345-192.168.203.56@tcp:101:mrename.0 00010000:00010000:2.0:1787826373.696453:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826373.696487:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4c829 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c7c expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696556:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696570:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696582:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696593:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696600:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696611:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696703:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c83e lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696712:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c83e lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696719:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c83e lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826373.696773:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dc3:0x0]: returning size 6 00000004:00200000:2.0:1787826373.696778:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826373.696799:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c83e lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696813:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c83e lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696822:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696829:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696835:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.696841:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697026:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826373.697031:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697039:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697049:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826373.697051:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826373.697054:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697058:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c830 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697071:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697074:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697078:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697081:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826373.697082:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697085:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697092:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f7967ac1200) 00010000:00010000:2.0:1787826373.697093:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826373.697095:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697098:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4c837 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697156:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4c829 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c7c expref: 10 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697166:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c83e lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c7c expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.697170:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f784abfef00) 00010000:00010000:2.0:1787826373.697171:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826373.697173:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826373.697242:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226877:x1874663332698496:12345-192.168.203.56@tcp:101:mrename.0 Request processed in 814us (1024us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.697249:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68721 00000100:00100000:3.0:1787826373.701188:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826373.701252:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826373.701401:0:6540:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332698880 00000100:00100000:0.0:1787826373.701411:0:6540:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332698880 00000100:00100000:0.0:1787826373.701478:0:6540:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 1238 00000100:00100000:0.0:1787826373.701498:0:6540:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+9:226877:x1874663332698880:12345-192.168.203.56@tcp:36:mrename.0 00010000:00010000:0.0:1787826373.701556:0:6540:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826373.701585:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4c83e lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c7c expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.701608:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f784abfef00) 00010000:00010000:0.0:1787826373.701619:0:6540:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826373.701701:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837bb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346354944:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.701758:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.701783:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332698624 00000100:00100000:2.0:1787826373.701789:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332698624 00000100:00100000:0.0:1787826373.701820:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ecc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826373.701855:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11230 00000100:00100000:2.0:1787826373.701866:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332698624:12345-192.168.203.56@tcp:103:mrename.0 00010000:00010000:2.0:1787826373.701876:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826373.701892:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42000/0xe21309aa81f4c822 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c75 expref: 7 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.701914:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f794b57a000) 00010000:00010000:2.0:1787826373.701922:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826373.701974:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332698624:12345-192.168.203.56@tcp:103:mrename.0 Request processed in 109us (799us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.701981:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11230 00000100:00100000:1.0:1787826373.702137:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346354944 00000100:00100000:1.0:1787826373.702142:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346354944 00000100:00100000:1.0:1787826373.702180:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24191 00000100:00100000:1.0:1787826373.702188:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346354944:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.702278:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346354944:12345-0@lo:1000:mdt_io00_000.0 Request processed in 91us (519us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.702285:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24191 00000100:00100000:0.0:1787826373.704669:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837bb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346354944:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.704780:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837bb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355072:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.706471:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826373.706634:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.708420:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346355072 00000100:00100000:1.0:1787826373.708436:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346355072 00000100:00100000:1.0:1787826373.708511:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24192 00000100:00100000:1.0:1787826373.708522:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355072:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.708667:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355072:12345-0@lo:1000:mdt_io00_000.0 Request processed in 147us (2203us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.708678:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24192 00000100:00100000:2.0:1787826373.708811:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837bb80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355072:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826373.708873:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19d:0x0] 00000100:00100000:2.0:1787826373.708933:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355200:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.708979:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.709020:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.709055:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346355200 00000100:00100000:1.0:1787826373.709061:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346355200 00000100:00100000:1.0:1787826373.709331:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24193 00000100:00100000:1.0:1787826373.709344:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355200:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.709463:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355200:12345-0@lo:1000:mdt_io00_000.0 Request processed in 122us (483us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.709470:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24193 00000100:00100000:0.0:1787826373.712638:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2680 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355200:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787826373.712698:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5e:0x0] 00000100:00100000:0.0:1787826373.713617:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355328:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.713687:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826373.713778:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.716364:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346355328 00000100:00100000:1.0:1787826373.716380:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346355328 00000100:00100000:1.0:1787826373.716447:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24194 00000100:00100000:1.0:1787826373.716458:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355328:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.716627:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355328:12345-0@lo:1000:mdt_io00_000.0 Request processed in 170us (2939us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.716634:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24194 00000100:00100000:3.0:1787826373.716798:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355328:0@lo:1000:mdt_io00_000.0 00000004:00080000:3.0:1787826373.716882:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19d:0x0] 00000100:00100000:3.0:1787826373.716940:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355456:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.716985:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826373.717024:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.717343:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346355456 00000100:00100000:1.0:1787826373.717349:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346355456 00000100:00100000:1.0:1787826373.717390:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24195 00000100:00100000:1.0:1787826373.717399:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355456:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.717471:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355456:12345-0@lo:1000:mdt_io00_000.0 Request processed in 73us (485us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.717478:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24195 00000100:00100000:1.0:1787826373.717522:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355456:0@lo:1000:mdt_io00_000.0 00000004:00020000:1.0:1787826373.717559:0:6540:0:(mdt_reint.c:2937:mdt_reint_rename()) cfs_fail_race id 16a waking 00010000:00010000:1.0:1787826373.717606:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.717620:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.717629:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826373.717641:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784ade3b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355584:0@lo:101:mdt_io00_000.0 00000100:00100000:1.0:1787826373.717667:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826373.717689:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78507053c0 going to sleep for 11 seconds 00000004:00020000:1.0:1787826373.717702:0:11629:0:(mdt_open.c:1594:mdt_reint_open()) cfs_fail_race id 16a awake: rc=4434 00010000:00010000:1.0:1787826373.717761:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.717774:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.717786:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826373.717799:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1dc3:0x0]: returning size 6 00010000:00010000:1.0:1787826373.718046:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718055:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718060:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718068:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718077:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826373.718081:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718089:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718257:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:1.0:1787826373.718262:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826373.718267:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718275:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c7c7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718291:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718297:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718303:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718308:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826373.718312:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718317:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718328:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f7967ac1200) 00010000:00010000:1.0:1787826373.718331:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826373.718333:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 2/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718339:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c7ce lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718392:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c7c0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718405:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 11 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.718411:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826373.718414:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826373.718416:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826373.718472:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226875:x1874663332697856:12345-192.168.203.56@tcp:101:cat.0 Request processed in 566798us (567212us total) trans 8589957499 rc 0/0 00000100:00100000:3.0:1787826373.718584:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346355584 00000100:00100000:3.0:1787826373.718590:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346355584 00000100:00100000:3.0:1787826373.718625:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24196 00000100:00100000:3.0:1787826373.718632:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:6540:x1874663346355584:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787826373.718644:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826373.718659:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c85a lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c845 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.718679:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.718688:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c85a lrc: 4/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c845 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.718714:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 6 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.718725:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 6 pid: 17813 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826373.718738:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346355712:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826373.718783:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.718997:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68716 00000100:00100000:0.0:1787826373.728296:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826373.728495:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332699008 00000100:00100000:1.0:1787826373.728505:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332699008 00010000:00010000:1.0:1787826373.728556:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 7 pid: 17813 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826373.728601:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22081 00000100:00100000:1.0:1787826373.728611:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2396:x1874663332699008:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826373.728645:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826373.728654:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 7 pid: 17813 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728666:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 7 pid: 17813 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728690:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:1.0:1787826373.728696:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c85a lrc: 3/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c845 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728708:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c85a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4c845 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728716:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c85a lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0xe21309aa81f4c845 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728749:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c85a lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c845 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826373.728766:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6512:1874663346355840:0@lo:105: 00000100:00100000:1.0:1787826373.728821:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826373.728862:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.728880:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346355840 00000100:00100000:1.0:1787826373.728883:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346355840 00000100:00100000:1.0:1787826373.728928:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7933 00000100:00100000:1.0:1787826373.728935:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b752680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6512:x1874663346355840:12345-0@lo:105: 00010000:00010000:1.0:1787826373.728948:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728958:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728966:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 4/0,1 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728974:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728981:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.728989:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 5/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.729004:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf200) 00000100:00100000:1.0:1787826373.729041:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b752680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6512:x1874663346355840:12345-0@lo:105: Request processed in 106us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.729047:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7933 00000100:00100000:1.0:1787826373.729089:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6512:1874663346355840:0@lo:105: 00010000:00010000:1.0:1787826373.729123:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826373.729163:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2396:x1874663332699008:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 553us (878us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.729171:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22081 00010000:00010000:3.0:1787826373.730186:0:125377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c7ab lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c4b expref: 6 pid: 17813 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826373.730209:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001_UUID:125377:1874663346355712:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826373.730247:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c85a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c845 expref: 7 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.730257:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826373.730259:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826373.730263:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826373.730326:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346355584:12345-0@lo:101:mdt_io00_000.0 Request processed in 11694us (12658us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826373.730334:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24196 00000100:00100000:1.0:1787826373.730702:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784ade3b80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355584:0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787826373.730721:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826373.730759:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19d:0x0] 00010000:00010000:1.0:1787826373.730822:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.730831:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.730840:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826373.731309:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355968:0@lo:101:mdt_io00_000.0 00000100:00100000:1.0:1787826373.731357:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826373.731397:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78507053c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826373.731449:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346355968 00000100:00100000:3.0:1787826373.731457:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346355968 00000100:00100000:3.0:1787826373.731516:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24197 00000100:00100000:3.0:1787826373.731527:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6540:x1874663346355968:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787826373.731542:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826373.731561:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c868 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c861 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.731586:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c868 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c861 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.731600:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c868 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c861 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.731609:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x4ec94fd6 (ffff9f79460a2b00) 00010000:00010000:3.0:1787826373.731613:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826373.731615:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68c00, rc 0) 00000100:00100000:3.0:1787826373.731661:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346355968:12345-0@lo:101:mdt_io00_000.0 Request processed in 135us (303us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826373.731667:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24197 00000100:00100000:0.0:1787826373.731828:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346355968:0@lo:101:mdt_io00_000.0 00010000:00010000:0.0:1787826373.731854:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c868 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.731873:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c868 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826373.731903:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19d:0x0] 00010000:00010000:0.0:1787826373.731923:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.731935:0:6540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.731944:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 4/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.731971:0:6540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 10 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.731986:0:6540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 10 pid: 35095 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826373.731998:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848379f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346356096:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826373.732070:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.743174:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848379f80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346356096:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826373.743212:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826373.746778:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826373.746899:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332699136 00000100:00100000:2.0:1787826373.746909:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332699136 00010000:00010000:2.0:1787826373.746952:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 11 pid: 35095 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826373.746991:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22082 00000100:00100000:2.0:1787826373.747001:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c81050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332699136:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826373.747014:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826373.747021:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 11 pid: 35095 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.747035:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 11 pid: 35095 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.747056:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c7b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c52 expref: 11 pid: 35095 timeout: 8285 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.747074:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826373.747079:0:8459:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.747089:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.747096:0:8459:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.747176:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826373.747235:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c81050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2394:x1874663332699136:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 234us (467us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.747242:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22082 00010000:00010000:0.0:1787826373.748134:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.748148:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.748177:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.748190:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.748199:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826373.748285:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356224:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.748346:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826373.748428:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ee40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826373.755826:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826373.756141:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332699264 00000100:00100000:3.0:1787826373.756153:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332699264 00000100:00100000:3.0:1787826373.756231:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30897 00000100:00100000:3.0:1787826373.756245:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332699264:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826373.756266:0:35352:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826373.756286:0:35352:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c87d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c83 expref: -99 pid: 35352 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826373.756348:0:35352:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346356352:192.168.203.56@tcp:106: 00000100:00100000:3.0:1787826373.756436:0:35352:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318f00 going to sleep for 11 seconds 00000100:00080000:0.0:1787826373.756810:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826373.757065:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346356224 00000100:00100000:1.0:1787826373.757072:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346356224 00000100:00100000:1.0:1787826373.757132:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24198 00000100:00100000:1.0:1787826373.757142:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356224:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.768445:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356224:12345-0@lo:1000:mdt_io00_000.0 Request processed in 11293us (20087us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.768462:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24198 00000100:00100000:0.0:1787826373.768777:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356224:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.768880:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848378380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356480:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.769058:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826373.769093:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ee40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826373.769201:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346356480 00000100:00100000:3.0:1787826373.769208:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346356480 00000100:00100000:3.0:1787826373.769265:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24199 00000100:00100000:3.0:1787826373.769275:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356480:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.769364:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356480:12345-0@lo:1000:mdt_io00_000.0 Request processed in 91us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826373.769371:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24199 00000100:00100000:2.0:1787826373.781144:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848378380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356480:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.781260:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356608:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.781331:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.781367:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826373.783641:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346356608 00000100:00100000:0.0:1787826373.783658:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346356608 00000100:00100000:0.0:1787826373.783741:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24200 00000100:00100000:0.0:1787826373.783756:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356608:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.783959:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356608:12345-0@lo:1000:mdt_io00_000.0 Request processed in 205us (2625us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826373.783968:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24200 00000100:00100000:2.0:1787826373.784358:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356608:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.784501:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356736:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.784547:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.784678:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362540 going to sleep for 11 seconds 00002000:00010000:3.0:1787826373.789331:0:35352:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x2c0000401:0x524:0x0] updating lvb size: 0 -> 6 00002000:00010000:3.0:1787826373.789339:0:35352:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x2c0000401:0x524:0x0] updating lvb mtime: 0 -> 1787826366 00002000:00010000:3.0:1787826373.789341:0:35352:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x2c0000401:0x524:0x0] updating lvb atime: 0 -> 1787826366 00002000:00010000:3.0:1787826373.789343:0:35352:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x2c0000401:0x524:0x0] updating lvb ctime: 0 -> 1787826366 00002000:00010000:3.0:1787826373.789345:0:35352:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x2c0000401:0x524:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:3.0:1787826373.789373:0:35352:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:35352:1874663346356352:192.168.203.56@tcp:106: 00010000:00010000:3.0:1787826373.789415:0:35352:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c87d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c83 expref: 6 pid: 35352 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.789443:0:35352:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c87d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c83 expref: 6 pid: 35352 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.789457:0:35352:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826373.789718:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332699264:12345-192.168.203.56@tcp:101:cat.0 Request processed in 33476us (33900us total) trans 0 rc 301/301 00000100:00100000:3.0:1787826373.789727:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30897 00000100:00100000:1.0:1787826373.791608:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346356736 00000100:00100000:1.0:1787826373.791622:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346356736 00000100:00100000:1.0:1787826373.795313:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24201 00000100:00100000:1.0:1787826373.795328:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356736:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.795487:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356736:12345-0@lo:1000:mdt_io00_000.0 Request processed in 162us (10938us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.795496:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24201 00000100:00100000:3.0:1787826373.795728:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356736:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.795857:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356864:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.795906:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826373.795930:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826373.796249:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346356864 00000100:00100000:2.0:1787826373.796257:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346356864 00000100:00100000:2.0:1787826373.796299:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24202 00000100:00100000:2.0:1787826373.796308:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356864:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.796386:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346356864:12345-0@lo:1000:mdt_io00_000.0 Request processed in 80us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.796394:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24202 00000100:00100000:2.0:1787826373.796425:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6c380 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346356864:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826373.796451:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5e:0x0] 00010000:00010000:2.0:1787826373.796554:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.796693:0:6540:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.796703:0:6540:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.796733:0:6540:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.796747:0:6540:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 9 pid: 11629 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826373.796759:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346356992:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826373.796819:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483626c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826373.803710:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:1.0:1787826373.806697:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826373.814476:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332699392 00000100:00100000:2.0:1787826373.814493:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332699392 00000100:00100000:3.0:1787826373.814825:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30898 00000100:00100000:3.0:1787826373.814836:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:226875:x1874663332699392:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:3.0:1787826373.814855:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826373.814878:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c8a expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.814911:0:8445:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 5 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.814920:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x524:0x0].0x0 since=6s v=(0 0) 00010000:00010000:3.0:1787826373.814925:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826373.814930:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.814968:0:8445:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 5 pid: 8444 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.814981:0:8445:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 5 pid: 8444 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826373.814996:0:8445:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8445:1874663346357120:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826373.815095:0:8445:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826373.819013:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826373.819178:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6540:1874663346356992:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826373.819224:0:6540:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826373.819455:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332699520 00000100:00100000:1.0:1787826373.819462:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332699520 00010000:00010000:1.0:1787826373.819516:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 10 pid: 11629 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826373.819556:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22083 00000100:00100000:1.0:1787826373.819685:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:2393:x1874663332699520:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826373.819703:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826373.819710:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 10 pid: 11629 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.819723:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 10 pid: 11629 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.819747:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c84c lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c59 expref: 10 pid: 11629 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.819765:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f7967ac1d00) 00010000:00010000:1.0:1787826373.819770:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.819780:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.819788:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.819820:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826373.819872:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2393:x1874663332699520:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 307us (869us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.819879:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22083 00010000:00010000:3.0:1787826373.819949:0:6540:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.819958:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.820024:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.820032:0:6540:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.820041:0:6540:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826373.820060:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357248:0@lo:101:mdt_io00_000.0 00000100:00100000:3.0:1787826373.820161:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826373.820189:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.820730:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346357248 00000100:00100000:1.0:1787826373.820739:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346357248 00000100:00100000:1.0:1787826373.820792:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24203 00000100:00100000:1.0:1787826373.820807:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6540:x1874663346357248:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787826373.820825:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826373.820845:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c899 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c892 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.820869:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c899 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c892 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.820882:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c899 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c892 expref: 9 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.820890:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f7967ac1200) 00010000:00010000:1.0:1787826373.820893:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1787826373.820896:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:1.0:1787826373.820952:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357248:12345-0@lo:101:mdt_io00_000.0 Request processed in 151us (790us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.820959:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24203 00000100:00100000:1.0:1787826373.820997:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c96a00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357248:0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787826373.821013:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c899 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826373.821024:0:6540:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c899 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826373.821046:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5f:0x0] 00000100:00100000:1.0:1787826373.821101:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357376:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.821384:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826373.821410:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78507059c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826373.821523:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346357376 00000100:00100000:3.0:1787826373.821529:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346357376 00000100:00100000:3.0:1787826373.821568:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24204 00000100:00100000:3.0:1787826373.821576:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784ade3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357376:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.821662:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784ade3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357376:12345-0@lo:1000:mdt_io00_000.0 Request processed in 86us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826373.821669:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24204 00000100:00100000:2.0:1787826373.821704:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357376:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826373.821738:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19d:0x0] 00000100:00100000:2.0:1787826373.821772:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357504:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.821801:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.821827:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.821874:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346357504 00000100:00100000:1.0:1787826373.821878:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346357504 00000100:00100000:1.0:1787826373.821909:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24205 00000100:00100000:1.0:1787826373.821917:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b752d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357504:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.821984:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b752d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357504:12345-0@lo:1000:mdt_io00_000.0 Request processed in 68us (182us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.821991:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24205 00000100:00100000:2.0:1787826373.822092:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357504:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826373.822326:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5e:0x0] 00010000:00010000:2.0:1787826373.822365:0:6540:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.822377:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.822386:0:6540:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826373.822431:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357632:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.822458:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.822550:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826373.822704:0:8445:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daa9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8445:1874663346357120:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826373.822733:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 6 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.822754:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:0.0:1787826373.822808:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226875:x1874663332699392:12345-192.168.203.56@tcp:101:cat.0 Request processed in 7973us (19109us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826373.822810:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346357632 00000100:00100000:3.0:1787826373.822816:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346357632 00000100:00100000:0.0:1787826373.822817:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30898 00000100:00100000:3.0:1787826373.822855:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24206 00000100:00100000:3.0:1787826373.822864:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357632:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.823061:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826373.823098:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357632:12345-0@lo:1000:mdt_io00_000.0 Request processed in 235us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826373.823123:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24206 00000100:00100000:2.0:1787826373.823406:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357632:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826373.823436:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5e:0x0] 00000100:00100000:2.0:1787826373.823489:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357760:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.823521:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.823546:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.823885:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346357760 00000100:00100000:1.0:1787826373.823891:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346357760 00000100:00100000:1.0:1787826373.823933:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24207 00000100:00100000:1.0:1787826373.823941:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357760:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.824018:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357760:12345-0@lo:1000:mdt_io00_000.0 Request processed in 77us (495us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.824025:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24207 00000100:00100000:2.0:1787826373.826834:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357760:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.826938:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357888:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.827007:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.827049:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.827265:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346357888 00000100:00100000:1.0:1787826373.827273:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346357888 00000100:00100000:1.0:1787826373.827328:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24208 00000100:00100000:1.0:1787826373.827339:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357888:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.827435:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346357888:12345-0@lo:1000:mdt_io00_000.0 Request processed in 98us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.827442:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24208 00000100:00100000:2.0:1787826373.828532:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346357888:0@lo:1000:mdt_io00_000.0 00000004:00080000:2.0:1787826373.828748:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19d:0x0] 00000100:00100000:2.0:1787826373.828805:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346358016:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.828860:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.828895:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.829491:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346358016 00000100:00100000:1.0:1787826373.829500:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346358016 00000100:00100000:1.0:1787826373.829544:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24209 00000100:00100000:1.0:1787826373.829553:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346358016:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.829839:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346358016:12345-0@lo:1000:mdt_io00_000.0 Request processed in 286us (977us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.829847:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24209 00000100:00100000:3.0:1787826373.829937:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b1880 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346358016:0@lo:1000:mdt_io00_000.0 00000004:00080000:3.0:1787826373.829967:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5e:0x0] 00000100:00100000:3.0:1787826373.830013:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346358144:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.830048:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826373.830072:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.830324:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346358144 00000100:00100000:1.0:1787826373.830329:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346358144 00000100:00100000:1.0:1787826373.830356:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24210 00000100:00100000:1.0:1787826373.830363:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346358144:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.830423:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346358144:12345-0@lo:1000:mdt_io00_000.0 Request processed in 60us (375us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.830430:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24210 00000100:00100000:3.0:1787826373.830514:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346358144:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.830551:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346358272:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.830586:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826373.830605:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.830805:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346358272 00000100:00100000:1.0:1787826373.830811:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346358272 00000100:00100000:1.0:1787826373.830843:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24211 00000100:00100000:1.0:1787826373.830850:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346358272:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787826373.830982:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346358272:12345-0@lo:1000:mdt_io00_000.0 Request processed in 131us (394us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826373.830992:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24211 00000100:00100000:3.0:1787826373.831708:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346358272:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.831803:0:6540:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c94000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346358400:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787826373.831841:0:6540:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826373.831871:0:6540:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826373.831999:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346358400 00000100:00100000:2.0:1787826373.832005:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346358400 00000100:00100000:2.0:1787826373.832044:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24212 00000100:00100000:2.0:1787826373.832052:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346358400:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787826373.832768:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+9:6540:x1874663346358400:12345-0@lo:1000:mdt_io00_000.0 Request processed in 714us (923us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.832776:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24212 00000100:00100000:3.0:1787826373.832869:0:6540:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c94000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6540:1874663346358400:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787826373.834071:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332699648 00000100:00100000:0.0:1787826373.834081:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332699648 00010000:00010000:0.0:1787826373.834319:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 6 pid: 8444 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826373.834362:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22084 00000100:00100000:0.0:1787826373.834372:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332699648:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826373.834388:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826373.834408:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 6 pid: 8444 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.834422:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 6 pid: 8444 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.834447:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000820f688e/0xe21309aa81f4c734 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c21 expref: 6 pid: 8444 timeout: 8286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.834464:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=0s v=(1 0) 00000100:00080000:1.0:1787826373.834640:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:3.0:1787826373.834775:0:6540:0:(mdd_dir.c:2033:mdd_finish_unlink()) Object [0x200000403:0x1dc3:0x0] is inserted into orphan list, open count = 1 00000040:00080000:3.0:1787826373.834857:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000bd0:0x2:0x0].945, 1240 off889744 00000040:00080000:3.0:1787826373.835704:0:6540:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000404:0x2:0x0].948, 1320 off989768 00000004:00080000:3.0:1787826373.835714:0:6540:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f7846318240 with version 1843 00010000:00010000:0.0:1787826373.836031:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787826373.836051:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.836063:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.836139:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826373.836158:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6512:1874663346358528:192.168.203.56@tcp:105: 00000100:00100000:0.0:1787826373.836271:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e6c0 going to sleep for 11 seconds 00000004:00080000:2.0:1787826373.837178:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1843 rpc_version 1843 00000020:00100000:2.0:1787826373.837230:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826373.837241:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7847b58e00 x1874663346358784/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2920/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826373.837262:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346358784:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826373.837330:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.837368:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826373.837537:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346358784 00000100:00100000:1.0:1787826373.837545:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346358784 00000100:00100000:1.0:1787826373.837613:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24213 00000100:00100000:1.0:1787826373.837622:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:30909:x1874663346358784:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787826373.837957:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:30909:x1874663346358784:12345-0@lo:1000:osp_up1-0.0 Request processed in 335us (625us total) trans 4294985497 rc 0/0 00000100:00100000:1.0:1787826373.837965:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24213 00000100:00100000:2.0:1787826373.838815:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346358784:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787826373.838931:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.838942:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.838948:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.838954:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826373.838958:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.838965:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.838979:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f7967ac1d00) 00010000:00010000:3.0:1787826373.838982:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826373.838986:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.838991:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4c884 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1dc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826373.839020:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1e5f:0x0] 00010000:00010000:3.0:1787826373.839027:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c899 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839035:0:6540:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c899 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839047:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839053:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839058:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839064:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826373.839066:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839073:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839080:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:3.0:1787826373.839083:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826373.839086:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839091:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4c86f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826373.839168:0:6540:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e2a0ca61 reply state 00000000f63a731c transno 8589957500 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839181:0:6540:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839188:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f794b57a700) 00010000:00010000:3.0:1787826373.839190:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826373.839193:0:6540:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00080000:3.0:1787826373.839197:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19d:0x0] 00010000:00010000:3.0:1787826373.839201:0:6540:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839208:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839224:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839240:0:6540:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826373.839244:0:6540:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0x19d:0x0] 00010000:00010000:3.0:1787826373.839249:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c868 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839255:0:6540:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c868 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839263:0:6540:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839269:0:6540:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839275:0:6540:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839280:0:6540:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826373.839283:0:6540:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839289:0:6540:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839297:0:6540:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffff9f784abfe600) 00010000:00010000:3.0:1787826373.839299:0:6540:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826373.839302:0:6540:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.839308:0:6540:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4c8a0 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826373.839391:0:6540:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adfa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+10:226877:x1874663332698880:12345-192.168.203.56@tcp:36:mrename.0 Request processed in 137893us (138138us total) trans 8589957500 rc 0/0 00000100:00100000:3.0:1787826373.839398:0:6540:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 1238 00010000:00010000:2.0:1787826373.839721:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826373.839728:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.839736:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf200 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:2.0:1787826373.839741:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826373.839748:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.839781:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.839788:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826373.839817:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 1/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.839825:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf200/0xe21309aa81f4c845 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c85a expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.839841:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826373.839843:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826373.839845:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826373.840304:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346358912:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826373.840355:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826373.840587:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346358912 00000100:00100000:2.0:1787826373.840595:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346358912 00000100:00100000:2.0:1787826373.840646:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11231 00000100:00100000:2.0:1787826373.840656:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3653:x1874663346358912:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826373.840669:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826373.840688:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69400/0xe21309aa81f4c85a lrc: 0/0,0 mode: --/CW res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c845 expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.840709:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f79460a2e00) 00010000:00010000:2.0:1787826373.840717:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826373.840757:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3653:x1874663346358912:12345-0@lo:103:ldlm_bl.0 Request processed in 103us (401us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.840764:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11231 00000100:00100000:2.0:1787826373.840802:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346358912:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787826373.842379:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6512:1874663346358528:192.168.203.56@tcp:105: 00010000:00010000:0.0:1787826373.842404:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826373.842452:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332699648:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 8081us (19390us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826373.842462:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22084 00000100:00100000:3.0:1787826373.843463:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826373.844001:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332699776 00000100:00100000:2.0:1787826373.844011:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332699776 00000100:00100000:2.0:1787826373.844072:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68722 00000100:00100000:2.0:1787826373.844083:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226875:x1874663332699776:12345-192.168.203.56@tcp:101:cat.0 00010000:00010000:2.0:1787826373.844289:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826373.844317:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c8a7 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c91 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.844379:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c8a7 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c91 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.844393:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c8a7 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c91 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826373.844411:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f784abfe300) 00010000:00010000:2.0:1787826373.844414:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787826373.844418:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef000, rc 0) 00000100:00100000:2.0:1787826373.844678:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226875:x1874663332699776:12345-192.168.203.56@tcp:101:cat.0 Request processed in 595us (1218us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826373.844690:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68722 00000100:00100000:3.0:1787826373.857568:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826373.858629:0:8450:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332699904 00000100:00100000:1.0:1787826373.858642:0:8450:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332699904 00002000:00100000:1.0:1787826373.858705:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x524:0x0] (0->4095) req@ffff9f78508c2300 x1874663332699904/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:729/0 lens 488/0 e 0 to 0 dl 1787826384 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826373.858717:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f78508c2300 with x1874663332699904 ext(0->4095) 00002000:00010000:1.0:1787826373.858724:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f78508c2300 00000100:00100000:1.0:1787826373.858762:0:8450:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 27671 00000100:00100000:1.0:1787826373.858774:0:8450:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332699904:12345-192.168.203.56@tcp:3:cat.0 00000100:00100000:1.0:1787826373.858966:0:8450:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332699904:12345-192.168.203.56@tcp:3:cat.0 Request processed in 192us (1407us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1787826373.858974:0:8450:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 27671 00002000:00100000:1.0:1787826373.858982:0:8450:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x524:0x0] (0->4095) req@ffff9f78508c2300 x1874663332699904/t0(0) o3->398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp:729/0 lens 488/4536 e 0 to 0 dl 1787826384 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1787826373.858991:0:8450:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9f78508c2300 with x1874663332699904 ext(0->4095) 00002000:00010000:1.0:1787826373.858994:0:8450:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9f78508c2300 00000100:00100000:0.0:1787826373.869361:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826373.870045:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332700160 00000100:00100000:3.0:1787826373.870057:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332700160 00000100:00100000:3.0:1787826373.870163:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9403 00000100:00100000:3.0:1787826373.870175:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226875:x1874663332700160:12345-192.168.203.56@tcp:35:cat.0 00000004:00080000:3.0:1787826373.870426:0:6530:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1dc3:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1787826373.870515:0:6530:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].217, 64 off22080 00000004:00080000:3.0:1787826373.870527:0:6530:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8186082420374ns, next at 8187082417848ns, rc = 0 00010000:00010000:3.0:1787826373.870576:0:6530:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.870591:0:6530:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.870603:0:6530:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.870610:0:6530:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.870616:0:6530:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.870622:0:6530:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826373.870641:0:6530:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1787826373.870743:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226875:x1874663332700160:12345-192.168.203.56@tcp:35:cat.0 Request processed in 570us (1392us total) trans 8589957501 rc 0/0 00000100:00100000:3.0:1787826373.870751:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9403 00010000:00010000:0.0:1787826373.871177:0:30445:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826373.871184:0:30445:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.871191:0:30445:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826373.871195:0:30445:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.871202:0:30445:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.871237:0:30445:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f784abfe300) 00010000:00010000:0.0:1787826373.871240:0:30445:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787826373.871244:0:30445:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.871250:0:30445:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8ae lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1dc3:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6530 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826373.871270:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826373.871272:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826373.871274:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826374.829557:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826375.292503:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826375.292633:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332700288 00000100:00100000:3.0:1787826375.292643:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332700288 00000100:00100000:3.0:1787826375.292730:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68723 00000100:00100000:3.0:1787826375.292744:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:226914:x1874663332700288:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826375.292763:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826375.292790:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8b5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c98 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.292866:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c8bc lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.292881:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c8bc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.292892:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c8bc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826375.292996:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5e:0x0]: returning size 4096 00000004:00200000:3.0:1787826375.293002:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787826375.293024:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c8bc lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.293052:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68800/0xe21309aa81f4c8b5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c98 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.293075:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c8bc lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c98 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.293083:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:3.0:1787826375.293086:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826375.293090:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68000, rc 0) 00000100:00100000:3.0:1787826375.293182:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226914:x1874663332700288:12345-192.168.203.56@tcp:101:rm.0 Request processed in 442us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826375.293189:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68723 00000100:00080000:1.0:1787826375.293484:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826375.296450:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.300300:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332700416 00000100:00100000:1.0:1787826375.300315:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332700416 00000100:00100000:1.0:1787826375.300402:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68724 00000100:00100000:1.0:1787826375.300413:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:226914:x1874663332700416:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826375.300433:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826375.300459:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c8c3 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801c9f expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.300530:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.300544:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.300556:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826375.300599:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x19d:0x0]: returning size 4096 00000004:00200000:1.0:1787826375.300605:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826375.300638:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c8c3 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x19d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 9 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.300655:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.300665:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:1.0:1787826375.300668:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826375.300671:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:1.0:1787826375.300758:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226914:x1874663332700416:12345-192.168.203.56@tcp:101:rm.0 Request processed in 346us (4317us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.300766:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68724 00000100:00100000:3.0:1787826375.303846:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.303946:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332700544 00000100:00100000:1.0:1787826375.303955:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332700544 00000100:00100000:1.0:1787826375.304025:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68725 00000100:00100000:1.0:1787826375.304036:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226914:x1874663332700544:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826375.304058:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826375.304080:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c8d1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801ca6 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.304161:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.304174:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.304184:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826375.304238:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x19b:0x0]: returning size 4096 00000004:00200000:1.0:1787826375.304243:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787826375.304290:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c8d1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 13 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.304307:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.304315:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826375.304318:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787826375.304321:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece800, rc 0) 00000100:00100000:1.0:1787826375.304382:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226914:x1874663332700544:12345-192.168.203.56@tcp:101:rm.0 Request processed in 349us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.304390:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68725 00000100:00080000:2.0:1787826375.304873:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826375.315019:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826375.315097:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332700800 00000100:00100000:2.0:1787826375.315130:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332700800 00000100:00100000:2.0:1787826375.315213:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68726 00000100:00100000:2.0:1787826375.315225:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:226914:x1874663332700800:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826375.315245:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826375.315282:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c8df lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cad expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826375.315439:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5e:0x0]: returning size 4096 00010000:00010000:2.0:1787826375.315559:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c8df lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cad expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.315575:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f784abfed00) 00010000:00010000:2.0:1787826375.315579:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826375.315584:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ee000/0xe21309aa81f4c8df lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cad expref: 10 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.315606:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee000, rc 0) 00000100:00100000:2.0:1787826375.315666:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226914:x1874663332700800:12345-192.168.203.56@tcp:101:rm.0 Request processed in 444us (656us total) trans 4294985498 rc 301/301 00000100:00100000:2.0:1787826375.315674:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68726 00000100:00100000:3.0:1787826375.332363:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826375.334301:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332700928 00000100:00100000:0.0:1787826375.334317:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332700928 00000100:00100000:0.0:1787826375.334399:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9404 00000100:00100000:0.0:1787826375.334413:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226914:x1874663332700928:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826375.349277:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226914:x1874663332700928:12345-192.168.203.56@tcp:37:rm.0 Request processed in 14857us (16912us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.349290:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9404 00000100:00100000:1.0:1787826375.357979:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.358019:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332701056 00000100:00100000:1.0:1787826375.358029:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332701056 00000100:00100000:1.0:1787826375.358092:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9405 00000100:00100000:1.0:1787826375.358118:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226914:x1874663332701056:12345-192.168.203.56@tcp:37:rm.0 00000100:00100000:1.0:1787826375.365151:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226914:x1874663332701056:12345-192.168.203.56@tcp:37:rm.0 Request processed in 7040us (7170us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.365165:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9405 00000100:00100000:1.0:1787826375.371769:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826375.371915:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332701312 00000100:00100000:2.0:1787826375.371928:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332701312 00000100:00100000:2.0:1787826375.371994:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9406 00000100:00100000:2.0:1787826375.372010:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967aef850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:226914:x1874663332701312:12345-192.168.203.56@tcp:35:rm.0 00000040:00080000:2.0:1787826375.372337:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1236, 344 off1129024 00000040:00080000:2.0:1787826375.372386:0:6531:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1555, 424 off1575656 00000004:00080000:2.0:1787826375.372393:0:6531:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f78513b3840 with version 6900 00000004:00080000:1.0:1787826375.372735:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6900 rpc_version 6900 00000020:00100000:1.0:1787826375.372768:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826375.372778:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784e5a5c00 x1874663346359168/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826375.372815:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346359168:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826375.372876:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826375.372913:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826375.373042:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346359168 00000100:00100000:0.0:1787826375.373049:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346359168 00000100:00100000:0.0:1787826375.373097:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24214 00000100:00100000:0.0:1787826375.373127:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346359168:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826375.373304:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346359168:12345-0@lo:1000:osp_up0-1.0 Request processed in 199us (426us total) trans 8589957502 rc 0/0 00000100:00100000:0.0:1787826375.373312:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24214 00000100:00100000:1.0:1787826375.373388:0:7471:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957498 gen 2 00000004:00080000:1.0:1787826375.373393:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957495 committed transno 8589957498 00000004:00080000:1.0:1787826375.373416:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957496 committed transno 8589957498 00000004:00080000:1.0:1787826375.373425:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957497 committed transno 8589957498 00000004:00080000:1.0:1787826375.373434:0:7471:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957498 committed transno 8589957498 00000100:00100000:1.0:1787826375.373445:0:7471:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9f784e5a5c00 x1874663346359168/t8589957502(8589957502) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787826391 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826375.373481:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346359168:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826375.373572:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967aef850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226914:x1874663332701312:12345-192.168.203.56@tcp:35:rm.0 Request processed in 1568us (1814us total) trans 4294985499 rc 0/0 00000100:00100000:3.0:1787826375.373583:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9406 00000100:00100000:1.0:1787826375.383878:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.383926:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332701568 00000100:00100000:1.0:1787826375.383936:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332701568 00000100:00100000:1.0:1787826375.384000:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68727 00000100:00100000:1.0:1787826375.384012:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:226914:x1874663332701568:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826375.384036:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826375.384065:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c8ed lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cb4 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.384270:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c8f4 lrc: 3/1,0 mode: --/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.384285:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c8f4 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.384296:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c8f4 lrc: 3/1,0 mode: CR/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826375.384307:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5e:0x0]: returning size 4096 00010000:00010000:1.0:1787826375.384407:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecea00/0xe21309aa81f4c8ed lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cb4 expref: 13 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.384427:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c8f4 lrc: 3/0,0 mode: CR/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cb4 expref: 12 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.384442:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:1.0:1787826375.384446:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1787826375.384450:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826375.384510:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226914:x1874663332701568:12345-192.168.203.56@tcp:101:rm.0 Request processed in 501us (642us total) trans 4294985500 rc 0/0 00000100:00100000:1.0:1787826375.384518:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68727 00000100:00100000:1.0:1787826375.396896:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.396946:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332701696 00000100:00100000:1.0:1787826375.396956:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332701696 00000100:00100000:1.0:1787826375.397024:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68728 00000100:00100000:1.0:1787826375.397037:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226914:x1874663332701696:12345-192.168.203.56@tcp:41:rm.0 00000004:00000024:1.0:1787826375.397843:0:17813:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 7808696320 avail 7345565696 00000100:00100000:1.0:1787826375.399048:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226914:x1874663332701696:12345-192.168.203.56@tcp:41:rm.0 Request processed in 2000us (2145us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.399631:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68728 00000100:00100000:2.0:1787826375.402341:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826375.402391:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332701824 00000100:00100000:3.0:1787826375.402401:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332701824 00000100:00100000:3.0:1787826375.402465:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68729 00000100:00100000:3.0:1787826375.402490:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226914:x1874663332701824:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826375.402513:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826375.402537:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c902 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cbb expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402609:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402626:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402638:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402652:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402659:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402666:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402727:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c917 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402746:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c917 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402754:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c917 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.402777:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c917 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402792:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402800:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 4/1,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402806:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402812:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402818:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826375.402823:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402830:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402843:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:3.0:1787826375.402846:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826375.402850:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402857:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4c909 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402868:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402874:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.402880:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.403086:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826375.403090:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.403097:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.403129:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f794b57a700) 00010000:00010000:3.0:1787826375.403132:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826375.403136:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.403142:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4c910 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.403181:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68800/0xe21309aa81f4c902 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cbb expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.403194:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c917 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cbb expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.403200:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f79460a2200) 00010000:00010000:3.0:1787826375.403217:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787826375.403219:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69a00, rc 0) 00000100:00100000:3.0:1787826375.403294:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226914:x1874663332701824:12345-192.168.203.56@tcp:101:rm.0 Request processed in 818us (962us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826375.403302:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68729 00000100:00100000:1.0:1787826375.409039:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826375.409168:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332701952 00000100:00100000:3.0:1787826375.409178:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332701952 00000100:00100000:3.0:1787826375.409247:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68730 00000100:00100000:3.0:1787826375.409260:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:226914:x1874663332701952:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826375.409284:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826375.409305:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c91e lrc: 2/0,0 mode: --/CR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cc2 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.409368:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c925 lrc: 3/1,0 mode: --/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.409382:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c925 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.409393:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c925 lrc: 3/1,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826375.409442:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5f:0x0]: returning size 6 00000004:00200000:3.0:1787826375.409447:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00010000:00010000:3.0:1787826375.409553:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c91e lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cc2 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.409570:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c925 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cc2 expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.409579:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f7967ac1200) 00010000:00010000:3.0:1787826375.409593:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787826375.409596:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826375.409665:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226914:x1874663332701952:12345-192.168.203.56@tcp:101:rm.0 Request processed in 409us (635us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826375.409671:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68730 00000100:00100000:0.0:1787826375.414493:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.414677:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332702080 00000100:00100000:1.0:1787826375.414685:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332702080 00000100:00100000:1.0:1787826375.414745:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11232 00000100:00100000:1.0:1787826375.414756:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:2393:x1874663332702080:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:1.0:1787826375.414772:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826375.414800:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69a00/0xe21309aa81f4c917 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cbb expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.414819:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f79460a2200) 00010000:00010000:1.0:1787826375.414832:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826375.414885:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332702080:12345-192.168.203.56@tcp:103:rm.0 Request processed in 130us (399us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.414892:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11232 00000100:00100000:2.0:1787826375.417015:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.417101:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332702336 00000100:00100000:1.0:1787826375.417194:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332702336 00000100:00100000:3.0:1787826375.417311:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68731 00000100:00100000:3.0:1787826375.417326:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226914:x1874663332702336:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826375.417403:0:35093:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826375.417434:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c925 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cc2 expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.417449:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f7967ac1200) 00010000:00010000:3.0:1787826375.417453:0:35093:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826375.417498:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.417511:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.417523:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826375.417548:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c3480 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346359296:0@lo:101:mdt00_010.0 00000100:00100000:3.0:1787826375.417614:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826375.417659:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826375.417721:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346359296 00000100:00100000:1.0:1787826375.417726:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346359296 00000100:00100000:1.0:1787826375.417784:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24215 00000100:00100000:1.0:1787826375.417792:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:35093:x1874663346359296:12345-0@lo:101:mdt00_010.0 00010000:00010000:1.0:1787826375.417807:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826375.417823:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c92c expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.417847:0:6532:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.417856:0:6532:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 4/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c92c expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.417879:0:6532:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.417892:0:6532:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 11 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.417904:0:6532:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346359424:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826375.417977:0:6532:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826375.421629:0:6532:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6532:1874663346359424:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826375.421673:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c92c expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.421687:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826375.421691:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826375.421696:0:6532:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 3/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c92c expref: 9 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.421706:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826375.421769:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346359296:12345-0@lo:101:mdt00_010.0 Request processed in 3976us (4152us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.421777:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24215 00000100:00100000:3.0:1787826375.422167:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c3480 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346359296:0@lo:101:mdt00_010.0 00010000:00010000:3.0:1787826375.422190:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 4/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.422212:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 4/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.422958:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826375.423161:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332702464 00000100:00100000:0.0:1787826375.423171:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332702464 00010000:00010000:0.0:1787826375.423338:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 12 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.423376:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22085 00000100:00100000:0.0:1787826375.423387:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2394:x1874663332702464:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826375.423409:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826375.423416:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 12 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.423429:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 12 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.423451:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4c8d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x19b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801ca6 expref: 12 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.423467:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:0.0:1787826375.423483:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c92c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.423493:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c92c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.423501:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c92c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.423532:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c92c expref: 8 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.423546:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346359552:0@lo:105: 00000100:00100000:0.0:1787826375.423596:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826375.423644:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826375.423711:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346359552 00000100:00100000:1.0:1787826375.423716:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346359552 00000100:00100000:1.0:1787826375.423753:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7934 00000100:00100000:1.0:1787826375.423760:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346359552:12345-0@lo:105: 00010000:00010000:1.0:1787826375.423771:0:6510:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 5/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.423780:0:6510:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 5/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.423789:0:6510:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 5/0,1 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.423797:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 5/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.423877:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.423887:0:6510:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 6/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.423911:0:6510:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68800) 00000100:00100000:1.0:1787826375.423965:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346359552:12345-0@lo:105: Request processed in 204us (368us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.423972:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7934 00010000:00010000:0.0:1787826375.424066:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.424069:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346359552:0@lo:105: 00010000:00010000:0.0:1787826375.424082:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.424092:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826375.424162:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2394:x1874663332702464:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 777us (1207us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.424171:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22085 00000004:00080000:0.0:1787826375.424300:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19b:0x0] 00010000:00010000:0.0:1787826375.424338:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.424346:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.424356:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.424374:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c15d880 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346359680:0@lo:101:mdt00_010.0 00000100:00100000:0.0:1787826375.424404:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826375.424431:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826375.424541:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346359680 00000100:00100000:1.0:1787826375.424546:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346359680 00000100:00100000:1.0:1787826375.424588:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24216 00000100:00100000:1.0:1787826375.424596:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346359680:12345-0@lo:101:mdt00_010.0 00010000:00010000:1.0:1787826375.424612:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826375.424627:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c93a expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.424646:0:8152:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 2/0,1 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.424655:0:8152:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 4/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c93a expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.424668:0:8152:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00080000:00080000:1.0:1787826375.424701:0:8152:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00010000:00010000:1.0:1787826375.424746:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 3/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c93a expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.424754:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f794b57a700) 00010000:00010000:1.0:1787826375.424757:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826375.424759:0:8152:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 3/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c93a expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.424766:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:0.0:1787826375.425330:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c15d880 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346359680:0@lo:101:mdt00_010.0 00010000:00010000:0.0:1787826375.425347:0:35093:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 4/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.425358:0:35093:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 4/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.425429:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346359680:12345-0@lo:101:mdt00_010.0 Request processed in 211us (400us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.425440:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24216 00010000:00010000:3.0:1787826375.427052:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c868 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.427071:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c868 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.427150:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c868 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.427158:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826375.427197:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7848ecf000/0xe21309aa81f4c861 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c868 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.427232:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:3.0:1787826375.427233:0:30894:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c899 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.427245:0:30894:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c899 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.427256:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346359808:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787826375.427265:0:30894:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c899 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.427272:0:30894:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826375.427279:0:30894:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffff9f7846e68600/0xe21309aa81f4c892 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c899 expref: -99 pid: 6540 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826375.427294:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957502 is committed 00000001:00080000:3.0:1787826375.427305:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957501 is committed 00000004:00080000:3.0:1787826375.427311:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 222 committed 00000100:00100000:0.0:1787826375.427330:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:3.0:1787826375.427338:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957500 is committed 00000100:00100000:0.0:1787826375.427376:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346359936:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787826375.427401:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826375.427443:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346359808 00000100:00100000:0.0:1787826375.427450:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346359808 00000100:00100000:0.0:1787826375.427493:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346359936 00000100:00100000:0.0:1787826375.427496:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346359936 00000100:00100000:0.0:1787826375.427520:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11233 00000100:00100000:0.0:1787826375.427529:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+10:3655:x1874663346359808:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826375.427542:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826375.427561:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68c00/0xe21309aa81f4c868 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19d:0x0].0x4ec94fd6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c861 expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.427578:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x4ec94fd6 (ffff9f79460a2b00) 00010000:00010000:0.0:1787826375.427586:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787826375.427622:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+9:3655:x1874663346359808:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 93us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826375.427630:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11233 00000100:00100000:0.0:1787826375.427651:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11234 00000100:00100000:0.0:1787826375.427659:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c81850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+8:3655:x1874663346359936:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787826375.427667:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826375.427736:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.427746:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.427752:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.427758:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826375.427762:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.427769:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.427793:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f794b57a700) 00010000:00010000:2.0:1787826375.427802:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826375.427806:0:3638:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c93a expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.427816:0:3638:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c93a expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.427824:0:3638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c93a expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.427850:0:3638:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c93a expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.427862:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0000_UUID:3638:1874663346360192:0@lo:105: 00000100:00100000:2.0:1787826375.427895:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826375.427935:0:3638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826375.427951:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346360064:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1787826375.427983:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826375.428024:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346360192 00000100:00100000:2.0:1787826375.428029:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346360192 00010000:00010000:0.0:1787826375.428056:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece000/0xe21309aa81f4c899 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c892 expref: 8 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.428067:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7935 00010000:00010000:0.0:1787826375.428070:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f7967ac1200) 00000100:00100000:2.0:1787826375.428077:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:3638:x1874663346360192:12345-0@lo:105: 00010000:00010000:2.0:1787826375.428088:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 5/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428133:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 5/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428141:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 5/0,1 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428149:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 5/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428156:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428164:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 6/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428196:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7845d43800) 00010000:00010000:0.0:1787826375.428238:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826375.428261:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:3638:x1874663346360192:12345-0@lo:105: Request processed in 185us (364us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.428271:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7935 00000100:00100000:0.0:1787826375.428277:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c81850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000-mdtlov_UUID+7:3655:x1874663346359936:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 618us (873us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826375.428286:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11234 00000100:00100000:2.0:1787826375.428309:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346360064 00000100:00100000:2.0:1787826375.428315:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346360064 00000100:00080000:3.0:1787826375.428356:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826375.428360:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346359936:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826375.428373:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30899 00000100:00100000:0.0:1787826375.428376:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346359808:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787826375.428386:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346360064:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1787826375.428414:0:17826:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1316 count 1 00010000:00010000:2.0:1787826375.428437:0:17826:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428463:0:17826:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428471:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=1s v=(1 0) 00010000:00010000:2.0:1787826375.428475:0:17826:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826375.428480:0:17826:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428504:0:17826:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 5 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428518:0:17826:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 5 pid: 8445 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.428530:0:17826:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346360320:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826375.428598:0:17826:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549a180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826375.428664:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_000:lustre-MDT0000_UUID:3638:1874663346360192:0@lo:105: 00010000:00010000:2.0:1787826375.428678:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.428685:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4c876 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6540 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.428753:0:35093:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.428763:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826375.428784:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19b:0x0] 00000100:00100000:1.0:1787826375.428822:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346360448:0@lo:1000:mdt00_010.0 00000100:00100000:1.0:1787826375.428858:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826375.428889:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826375.429179:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346360448 00000100:00100000:0.0:1787826375.429184:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346360448 00000100:00100000:0.0:1787826375.429234:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24217 00000100:00100000:0.0:1787826375.429244:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346360448:12345-0@lo:1000:mdt00_010.0 00000100:00100000:0.0:1787826375.429330:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346360448:12345-0@lo:1000:mdt00_010.0 Request processed in 87us (470us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826375.429338:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24217 00000100:00100000:2.0:1787826375.429493:0:35093:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957502 gen 2 00000004:00080000:2.0:1787826375.429499:0:35093:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957502 committed transno 8589957502 00000100:00100000:2.0:1787826375.429521:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346360448:0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826375.429572:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346360576:0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826375.429621:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826375.430232:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346360576 00000100:00100000:3.0:1787826375.430240:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346360576 00000100:00100000:2.0:1787826375.430258:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826375.430284:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24218 00000100:00100000:3.0:1787826375.430293:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346360576:12345-0@lo:1000:mdt00_010.0 00000100:00100000:3.0:1787826375.430396:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346360576:12345-0@lo:1000:mdt00_010.0 Request processed in 103us (774us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826375.430404:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24218 00000100:00100000:2.0:1787826375.430923:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346360576:0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826375.430992:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346360704:0@lo:1000:mdt00_010.0 00000100:00100000:2.0:1787826375.431026:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826375.431055:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7850705240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826375.431172:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346360704 00000100:00100000:0.0:1787826375.431178:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346360704 00000100:00100000:0.0:1787826375.431223:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24219 00000100:00100000:0.0:1787826375.431239:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346360704:12345-0@lo:1000:mdt00_010.0 00000100:00100000:0.0:1787826375.431406:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346360704:12345-0@lo:1000:mdt00_010.0 Request processed in 168us (378us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826375.431412:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24219 00000100:00100000:2.0:1787826375.431460:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5a5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346360704:0@lo:1000:mdt00_010.0 00010000:00010000:2.0:1787826375.431524:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.431533:0:35093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.431544:0:35093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.431556:0:35093:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346360832:0@lo:101:mdt00_010.0 00000100:00100000:2.0:1787826375.431579:0:35093:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826375.431683:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346360832 00000100:00100000:0.0:1787826375.431688:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346360832 00000100:00100000:2.0:1787826375.431691:0:35093:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784549ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826375.431721:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24220 00000100:00100000:0.0:1787826375.431740:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:35093:x1874663346360832:12345-0@lo:101:mdt00_010.0 00010000:00010000:0.0:1787826375.431749:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826375.431767:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c956 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c94f expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.431792:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c956 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c94f expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.431807:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c956 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c94f expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.431815:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f794b57a500) 00010000:00010000:0.0:1787826375.431818:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787826375.431822:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42000, rc 0) 00000100:00100000:0.0:1787826375.431859:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:35093:x1874663346360832:12345-0@lo:101:mdt00_010.0 Request processed in 120us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826375.431866:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24220 00000100:00100000:2.0:1787826375.431887:0:35093:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-mdtlov_UUID:35093:1874663346360832:0@lo:101:mdt00_010.0 00010000:00010000:2.0:1787826375.431901:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.431912:0:35093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.431945:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.431953:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.431961:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.432247:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826375.432316:0:17826:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:17826:1874663346360320:192.168.203.56@tcp:104: 00000040:00080000:2.0:1787826375.432329:0:35093:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].231, 64 off22976 00010000:00010000:0.0:1787826375.432342:0:17826:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826375.432345:0:35093:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 8187644238426ns, next at 8188644235308ns, rc = 0 00000040:00080000:2.0:1787826375.432377:0:35093:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1237, 384 off1129408 00000100:00100000:3.0:1787826375.432423:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332702592 00000040:00080000:2.0:1787826375.432428:0:35093:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1556, 464 off1576120 00000100:00100000:3.0:1787826375.432429:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332702592 00000004:00080000:2.0:1787826375.432438:0:35093:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784549ae40 with version 6901 00000004:00080000:1.0:1787826375.432530:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6901 rpc_version 6901 00000020:00100000:1.0:1787826375.432554:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00010000:00010000:3.0:1787826375.432555:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 6 pid: 8445 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.432567:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b19c000 x1874663346361088/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826375.432585:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22086 00000100:00100000:1.0:1787826375.432590:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346361088:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826375.432596:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332702592:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826375.432605:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826375.432609:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 6 pid: 8445 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.432618:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 6 pid: 8445 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.432635:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b030b3fe/0xe21309aa81f4c88b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400030020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c8a expref: 6 pid: 8445 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.432637:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826375.432651:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826375.432656:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x524:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826375.432662:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.432666:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483343c0 going to sleep for 11 seconds 00010000:00010000:3.0:1787826375.432673:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.432687:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346361088 00000100:00100000:1.0:1787826375.432693:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346361088 00010000:00010000:3.0:1787826375.432697:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826375.432729:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24221 00000100:00100000:1.0:1787826375.432738:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346361088:12345-0@lo:1000:osp_up0-1.0 00000100:00080000:0.0:1787826375.432790:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1787826375.432803:0:17826:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.432810:0:17826:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.432820:0:17826:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.432827:0:17826:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.432845:0:17826:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826375.432959:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346360064:12345-0@lo:6:osp-syn-1-0.0 Request processed in 4577us (4975us total) trans 4294976175 rc 0/0 00000100:00100000:0.0:1787826375.432965:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30899 00010000:00010000:0.0:1787826375.432984:0:6514:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826375.432990:0:6514:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.432996:0:6514:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef800 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826375.432999:0:6514:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.433010:0:6514:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.433020:0:6514:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.433025:0:6514:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008673aa3a/0xe21309aa81f4c948 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 17826 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.433040:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826375.433040:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346361088:12345-0@lo:1000:osp_up0-1.0 Request processed in 301us (403us total) trans 8589957503 rc 0/0 00010000:00010000:0.0:1787826375.433043:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826375.433045:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826375.433053:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24221 00000100:00100000:3.0:1787826375.433082:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332702592:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 488us (836us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826375.433092:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22086 00000004:00080000:2.0:1787826375.433178:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784d9fed80/2, rc 0, transno 4294976175 00000100:00100000:2.0:1787826375.433185:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346360064:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1787826375.433296:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346361088:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787826375.433347:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433360:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433368:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433376:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433382:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433389:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433403:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787826375.433423:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433429:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433435:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433440:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826375.433444:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433450:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433459:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:2.0:1787826375.433462:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826375.433465:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433471:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490eee00/0xe21309aa81f4c95d lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433483:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433495:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433502:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433524:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787826375.433536:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19b:0x0] 00010000:00010000:2.0:1787826375.433544:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433552:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433559:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433652:0:35093:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:3.0:1787826375.433654:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00000004:00080000:2.0:1787826375.433660:0:35093:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19b:0x0] 00010000:00010000:3.0:1787826375.433661:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433670:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.433675:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef200 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826375.433680:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.433684:0:35093:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.433691:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.433705:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f784abfe400) 00010000:00010000:3.0:1787826375.433708:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787826375.433712:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 1/0,0 mode: --/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.433718:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490ef200/0xe21309aa81f4c964 lrc: 0/0,0 mode: --/PW res: [0x240000403:0x1e5f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 35093 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.433752:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226914:x1874663332702336:12345-192.168.203.56@tcp:36:rm.0 Request processed in 16428us (16744us total) trans 4294985501 rc 0/0 00000100:00100000:2.0:1787826375.433764:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68731 00010000:00010000:3.0:1787826375.433807:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826375.433812:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.433818:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826375.433823:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.433830:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.433856:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.433866:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826375.433887:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826375.433891:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.433894:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.433902:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f78490ee800/0xe21309aa81f4c94f lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c956 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.433908:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68800 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:1.0:1787826375.433916:0:6515:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826375.433924:0:6515:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.433944:0:6515:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.433951:0:6515:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826375.433968:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 1/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.433975:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68800/0xe21309aa81f4c92c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c933 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.433989:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826375.433991:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826375.433993:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826375.434004:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b305c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346361344:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787826375.434037:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826375.434041:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826375.434045:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826375.434045:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826375.434089:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346361344 00000100:00100000:1.0:1787826375.434095:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346361344 00000100:00100000:1.0:1787826375.434167:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11235 00000100:00100000:1.0:1787826375.434176:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346361344:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826375.434187:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826375.434214:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4c933 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x19b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c92c expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.434228:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f7967ac1600) 00010000:00010000:1.0:1787826375.434235:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826375.434271:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3654:x1874663346361344:12345-0@lo:103:ldlm_bl.0 Request processed in 96us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.434278:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11235 00000100:00100000:1.0:1787826375.434305:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b305c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346361344:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826375.434370:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346361216:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826375.434396:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826375.434467:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346361216 00000100:00100000:1.0:1787826375.434471:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346361216 00000100:00100000:1.0:1787826375.434506:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11236 00000100:00100000:1.0:1787826375.434514:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3653:x1874663346361216:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787826375.434521:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826375.434533:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4c956 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c94f expref: 10 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.434544:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5f:0x0].0x0 (ffff9f794b57a500) 00010000:00010000:1.0:1787826375.434550:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826375.434598:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3653:x1874663346361216:12345-0@lo:103:ldlm_bl.0 Request processed in 84us (200us total) trans 0 rc 0/0 00010000:00010000:0.0:1787826375.434601:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826375.434606:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11236 00000100:00100000:3.0:1787826375.434708:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346361216:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826375.435923:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.435956:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826375.436007:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332702720 00000100:00100000:3.0:1787826375.436013:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332702720 00000100:00100000:2.0:1787826375.436238:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332702976 00000100:00100000:2.0:1787826375.436243:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332702976 00000100:00100000:2.0:1787826375.436295:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9407 00000100:00100000:2.0:1787826375.436305:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:226914:x1874663332702976:12345-192.168.203.56@tcp:35:rm.0 00000100:00100000:3.0:1787826375.436398:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11237 00000100:00100000:3.0:1787826375.436412:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:2394:x1874663332702720:12345-192.168.203.56@tcp:103:rm.0 00010000:00010000:3.0:1787826375.436425:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826375.436446:0:8459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4c78f lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c3d expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.436478:0:8459:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:3.0:1787826375.436483:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826375.436522:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332702720:12345-192.168.203.56@tcp:103:rm.0 Request processed in 111us (599us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.436523:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:226914:x1874663332702976:12345-192.168.203.56@tcp:35:rm.0 Request processed in 215us (559us total) trans 4294985502 rc 0/0 00000100:00100000:3.0:1787826375.436538:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11237 00000100:00100000:2.0:1787826375.436541:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9407 00000100:00100000:2.0:1787826375.439093:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826375.439165:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332703232 00000100:00100000:1.0:1787826375.439175:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332703232 00000100:00100000:1.0:1787826375.439248:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68732 00000100:00100000:1.0:1787826375.439262:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:226914:x1874663332703232:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826375.439383:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826375.439410:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ece200/0xe21309aa81f4c8f4 lrc: 0/0,0 mode: --/CR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cb4 expref: 14 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.439428:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68000/0xe21309aa81f4c8bc lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c98 expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.439439:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f78490efe00/0xe21309aa81f4c79d lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c44 expref: 12 pid: 8752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.439450:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:1.0:1787826375.439461:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826375.439566:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.439573:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.439579:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.439597:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346361472:0@lo:101:mdt00_011.0 00000100:00100000:1.0:1787826375.439637:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826375.439669:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483343c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826375.439686:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346361472 00000100:00100000:0.0:1787826375.439691:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346361472 00000100:00100000:3.0:1787826375.439830:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24222 00000100:00100000:3.0:1787826375.439841:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:35094:x1874663346361472:12345-0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826375.439856:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826375.439870:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c96b expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.439892:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.439901:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.439907:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 5/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.439925:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.439933:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 7 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826375.439941:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c0000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346361600:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826375.440001:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 7 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.440007:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 7 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826375.440013:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346361728:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826375.440027:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784631a840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826375.442725:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826375.442740:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c0000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346361600:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826375.442756:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346361728:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826375.442789:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.442802:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826375.442805:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826375.442809:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 10 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.442819:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69e00, rc 0) 00000100:00100000:3.0:1787826375.442882:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346361472:12345-0@lo:101:mdt00_011.0 Request processed in 3043us (3242us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826375.442892:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24222 00000100:00100000:0.0:1787826375.442938:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826375.443004:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332703360 00000100:00100000:2.0:1787826375.443010:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332703360 00010000:00010000:2.0:1787826375.443056:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 8 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.443078:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332703488 00000100:00100000:2.0:1787826375.443081:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332703488 00010000:00010000:2.0:1787826375.443093:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 8 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.443134:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22087 00000100:00100000:2.0:1787826375.443144:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332703360:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826375.443157:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826375.443163:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 8 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.443176:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 8 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.443216:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4c7dc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c60 expref: 8 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.443231:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826375.443240:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.443248:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787826375.443282:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22088 00000100:00100000:1.0:1787826375.443293:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:2394:x1874663332703488:12345-192.168.203.56@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826375.443302:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332703360:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 157us (584us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826375.443303:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787826375.443314:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22087 00000100:00080000:0.0:1787826375.443315:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787826375.443315:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 8 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.443333:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 8 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.443353:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c773 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c36 expref: 8 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.443367:0:6512:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826375.443372:0:6512:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.443387:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.443397:0:6512:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.443431:0:6512:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 9 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.443445:0:6512:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844295c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346361856:0@lo:105: 00000100:00100000:1.0:1787826375.443484:0:6512:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826375.443522:0:6512:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826375.443550:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307100 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346361472:0@lo:101:mdt00_011.0 00010000:00010000:1.0:1787826375.443565:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.443575:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826375.443828:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346361856 00000100:00100000:3.0:1787826375.443834:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346361856 00000100:00100000:3.0:1787826375.443868:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7936 00000100:00100000:3.0:1787826375.443874:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346361856:12345-0@lo:105: 00010000:00010000:3.0:1787826375.443884:0:6509:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.443893:0:6509:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.443902:0:6509:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.443915:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.443922:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.443929:0:6509:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.443955:0:6509:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7848ecf400) 00000100:00100000:3.0:1787826375.444038:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6512:x1874663346361856:12345-0@lo:105: Request processed in 165us (554us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826375.444173:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826375.444180:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7936 00010000:00010000:1.0:1787826375.444183:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826375.444215:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787826375.444252:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.444261:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.444270:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.444348:0:6512:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844295c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6512:1874663346361856:0@lo:105: 00010000:00010000:2.0:1787826375.444356:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826375.444384:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332703488:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 1094us (1445us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.444391:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22088 00000100:00100000:0.0:1787826375.444692:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844295500 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346361984:0@lo:101:mdt00_011.0 00000100:00100000:0.0:1787826375.444833:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826375.444865:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826375.444989:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346361984 00000100:00100000:1.0:1787826375.444996:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346361984 00000100:00100000:1.0:1787826375.445041:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24223 00000100:00100000:1.0:1787826375.445051:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:35094:x1874663346361984:12345-0@lo:101:mdt00_011.0 00010000:00010000:1.0:1787826375.445066:0:6532:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826375.445084:0:6532:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c980 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c979 expref: -99 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.445135:0:6532:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c980 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c979 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.445148:0:6532:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c980 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c979 expref: 11 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.445156:0:6532:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1700) 00010000:00010000:1.0:1787826375.445159:0:6532:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826375.445161:0:6532:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf600, rc 0) 00000100:00100000:1.0:1787826375.445218:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346361984:12345-0@lo:101:mdt00_011.0 Request processed in 155us (370us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.445226:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24223 00000100:00100000:3.0:1787826375.445376:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844295500 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346361984:0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826375.445392:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c980 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.445403:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c980 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787826375.445420:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787826375.445460:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346362112:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826375.445529:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826375.445556:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7977cfc600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826375.445579:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346362112 00000100:00100000:0.0:1787826375.445584:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346362112 00000100:00100000:0.0:1787826375.445617:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24224 00000100:00100000:0.0:1787826375.445625:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cf8b450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346362112:12345-0@lo:1000:mdt00_011.0 00000100:00100000:0.0:1787826375.445686:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cf8b450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346362112:12345-0@lo:1000:mdt00_011.0 Request processed in 62us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826375.445694:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24224 00000100:00100000:3.0:1787826375.445703:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346362112:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826375.445755:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346362240:0@lo:1000:mdt00_011.0 00000100:00100000:3.0:1787826375.445779:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826375.445798:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826375.445822:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346362240 00000100:00100000:1.0:1787826375.445827:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346362240 00000100:00100000:1.0:1787826375.445866:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24225 00000100:00100000:1.0:1787826375.445874:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346362240:12345-0@lo:1000:mdt00_011.0 00000100:00100000:1.0:1787826375.446067:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346362240:12345-0@lo:1000:mdt00_011.0 Request processed in 194us (285us total) trans 0 rc 1/1 00000100:00100000:1.0:1787826375.446076:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24225 00000100:00100000:3.0:1787826375.446182:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346362240:0@lo:1000:mdt00_011.0 00010000:00010000:3.0:1787826375.446237:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.446243:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.446249:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826375.446256:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346362368:0@lo:101:mdt00_011.0 00000100:00100000:3.0:1787826375.446274:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826375.446293:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826375.446306:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346362368 00000100:00100000:2.0:1787826375.446311:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346362368 00000100:00100000:2.0:1787826375.446338:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24226 00000100:00100000:2.0:1787826375.446343:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:35094:x1874663346362368:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826375.446349:0:125377:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826375.446359:0:125377:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c987 expref: -99 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.446373:0:125377:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 2/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.446379:0:125377:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 4/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c987 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.446396:0:125377:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.446404:0:125377:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 6 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.446411:0:125377:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346362496:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826375.446677:0:125377:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78556679c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826375.448924:0:125377:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:125377:1874663346362496:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826375.448964:0:125377:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c987 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.448975:0:125377:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:2.0:1787826375.448978:0:125377:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787826375.448982:0:125377:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 3/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c987 expref: 12 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.448993:0:125377:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826375.449044:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346362368:12345-0@lo:101:mdt00_011.0 Request processed in 2699us (2767us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.449057:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24226 00000100:00100000:3.0:1787826375.449094:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346362368:0@lo:101:mdt00_011.0 00010000:00010000:3.0:1787826375.449155:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.449167:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 4/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.451235:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826375.451283:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332703616 00000100:00100000:2.0:1787826375.451292:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332703616 00010000:00010000:2.0:1787826375.451336:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 7 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.451377:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22089 00000100:00100000:2.0:1787826375.451388:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c82850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332703616:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826375.451403:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826375.451411:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 4/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 7 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.451433:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 3/0,0 mode: PR/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 7 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.451549:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4c7f8 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c67 expref: 7 pid: 35094 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.451566:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:2.0:1787826375.451573:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c987 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.451583:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c987 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.451591:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xe21309aa81f4c987 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.451627:0:17812:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c987 expref: 11 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.451653:0:17812:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346362624:0@lo:105: 00000100:00100000:2.0:1787826375.451701:0:17812:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826375.451845:0:74129:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826375.451855:0:74129:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346362624 00000100:00100000:1.0:1787826375.451858:0:74129:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346362624 00000100:00100000:1.0:1787826375.451888:0:74129:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7937 00000100:00100000:1.0:1787826375.451894:0:74129:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d69850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346362624:12345-0@lo:105: 00010000:00010000:1.0:1787826375.451903:0:74129:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.451911:0:74129:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.451916:0:74129:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 5/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.451921:0:74129:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 5/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.451925:0:74129:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.451931:0:74129:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 6/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826375.451950:0:74129:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9f7846e68600) 00010000:00010000:3.0:1787826375.451980:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.451990:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.452001:0:17812:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78556679c0 going to sleep for 11 seconds 00010000:00010000:3.0:1787826375.452031:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 3/0,1 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.452042:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.452051:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.452071:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.452092:0:35094:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.452100:0:35094:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 4/0,1 mode: --/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826375.452101:0:74129:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d69850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:17812:x1874663346362624:12345-0@lo:105: Request processed in 207us (398us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.452126:0:74129:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7937 00010000:00010000:3.0:1787826375.452146:0:35094:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 11 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.452159:0:35094:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 11 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826375.452170:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001_UUID:35094:1874663346362752:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826375.452280:0:17812:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ddf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:17812:1874663346362624:0@lo:105: 00010000:00010000:2.0:1787826375.452297:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826375.452323:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c82850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332703616:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 938us (1098us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826375.452332:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784d02d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826375.452340:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22089 00000100:00100000:0.0:1787826375.453833:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001_UUID:35094:1874663346362752:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826375.453859:0:35094:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.454844:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826375.454883:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332703744 00000100:00100000:2.0:1787826375.454891:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332703744 00010000:00010000:2.0:1787826375.454924:0:125406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 12 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.454961:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22090 00000100:00100000:2.0:1787826375.454969:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:2393:x1874663332703744:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826375.454980:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826375.454985:0:125406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 12 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.454994:0:125406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 12 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.455013:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7848ecec00/0xe21309aa81f4c8ca lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c9f expref: 12 pid: 35080 timeout: 8287 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.455024:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:2.0:1787826375.455028:0:125406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,1 mode: --/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.455034:0:125406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.455038:0:125406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.455060:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826375.455102:0:35094:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826375.455131:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:2393:x1874663332703744:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 162us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.455142:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22090 00010000:00010000:0.0:1787826375.455150:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.455197:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.455219:0:35094:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.455229:0:35094:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.455254:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346362880:0@lo:101:mdt00_011.0 00000100:00100000:0.0:1787826375.455299:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826375.455338:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826375.455477:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346362880 00000100:00100000:2.0:1787826375.455482:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346362880 00000100:00100000:2.0:1787826375.455521:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24227 00000100:00100000:2.0:1787826375.455528:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:35094:x1874663346362880:12345-0@lo:101:mdt00_011.0 00010000:00010000:2.0:1787826375.455542:0:8152:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826375.455559:0:8152:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9aa lrc: 2/0,0 mode: --/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe21309aa81f4c9a3 expref: -99 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.455576:0:8152:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9aa lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xe21309aa81f4c9a3 expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.455592:0:8152:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9aa lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xe21309aa81f4c9a3 expref: 13 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.455599:0:8152:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfe500) 00010000:00010000:2.0:1787826375.455602:0:8152:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826375.455604:0:8152:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826375.455828:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346362880:12345-0@lo:101:mdt00_011.0 Request processed in 300us (529us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.455835:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24227 00000100:00100000:0.0:1787826375.456046:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346362880:0@lo:101:mdt00_011.0 00010000:00010000:0.0:1787826375.456062:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826375.456078:0:35094:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826375.456924:0:35094:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346363008:0@lo:403:mdt00_011.0 00000100:00100000:0.0:1787826375.456978:0:35094:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826375.457048:0:35094:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826375.457094:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346363008 00000100:00100000:1.0:1787826375.457101:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346363008 00000100:00100000:1.0:1787826375.457301:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24228 00000100:00100000:1.0:1787826375.457310:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346363008:12345-0@lo:403:mdt00_011.0 00000100:00100000:1.0:1787826375.459332:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:35094:x1874663346363008:12345-0@lo:403:mdt00_011.0 Request processed in 2015us (2344us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.459345:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24228 00000100:00100000:0.0:1787826375.459615:0:35094:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704e00 pname:cluuid:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-mdtlov_UUID:35094:1874663346363008:0@lo:403:mdt00_011.0 00000040:00080000:0.0:1787826375.460936:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].1238, 952 off1130360 00000040:00080000:0.0:1787826375.461219:0:35094:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].1557, 1032 off1577152 00000004:00080000:0.0:1787826375.461226:0:35094:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7848d5e3c0 with version 6902 00000004:00080000:1.0:1787826375.461314:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6902 rpc_version 6902 00000020:00100000:1.0:1787826375.461351:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826375.461363:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7844297b80 x1874663346363264/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826375.461386:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844297b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346363264:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826375.461450:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826375.461591:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826375.461645:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346363264 00000100:00100000:0.0:1787826375.461652:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346363264 00000100:00100000:0.0:1787826375.461737:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24229 00000100:00100000:0.0:1787826375.461747:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346363264:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787826375.462247:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346363264:12345-0@lo:1000:osp_up0-1.0 Request processed in 500us (794us total) trans 8589957504 rc 0/0 00000100:00100000:0.0:1787826375.462255:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24229 00000100:00100000:1.0:1787826375.462433:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844297b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346363264:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787826375.462507:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000403:0x1e5e:0x0]: returning size 4096 00010000:00010000:3.0:1787826375.462531:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462545:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462554:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462560:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826375.462565:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462572:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462588:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x19d:0x0].0x0 (ffff9f7967ac1a00) 00010000:00010000:3.0:1787826375.462591:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826375.462596:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 2/0,0 mode: --/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462602:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c99c lrc: 0/0,0 mode: --/EX res: [0x240000400:0x19d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462622:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462630:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462637:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462667:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787826375.462676:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x19b:0x0] 00010000:00010000:3.0:1787826375.462685:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 3/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462691:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462697:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462702:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826375.462705:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462711:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 4/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462719:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:3.0:1787826375.462722:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826375.462725:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462731:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7846e68200/0xe21309aa81f4c995 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462741:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462748:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 4/0,1 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462756:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462768:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787826375.462777:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826375.462782:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462789:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462795:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462809:0:35094:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787826375.462813:0:35094:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787826375.462820:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826375.462825:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462832:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826375.462836:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.462843:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462874:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462881:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826375.462896:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462903:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d42400/0xe21309aa81f4c9a3 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c9aa expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462917:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826375.462921:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 2/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462926:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826375.462929:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.462935:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 3/0,0 mode: EX/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462950:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 2/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462955:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826375.462971:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 1/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462983:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7846e68600/0xe21309aa81f4c987 lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c98e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.462995:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826375.462998:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.463004:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf400 already unused, calling callback (ffffffffc1986ee0) 00010000:00010000:3.0:1787826375.463008:0:7478:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.463014:0:7478:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.463030:0:7478:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.463035:0:7478:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826375.463042:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.463048:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf400/0xe21309aa81f4c96b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c972 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.463058:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826375.463060:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826375.463062:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826375.463073:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346363648:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826375.463140:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826375.463170:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826375.463177:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346363520:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787826375.463199:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826375.463244:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c980 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.463253:0:35094:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xe21309aa81f4c980 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826375.463320:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:226914:x1874663332703232:12345-192.168.203.56@tcp:36:rm.0 Request processed in 24061us (24234us total) trans 4294985503 rc 0/0 00000100:00100000:3.0:1787826375.463329:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68732 00010000:00010000:1.0:1787826375.463684:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826375.463693:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826375.463704:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346363392:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826375.463737:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787826375.463769:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826375.463988:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346363648 00000100:00100000:2.0:1787826375.463995:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346363648 00000100:00100000:2.0:1787826375.464033:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346363520 00000100:00100000:2.0:1787826375.464036:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346363520 00000100:00100000:2.0:1787826375.464055:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346363392 00000100:00100000:2.0:1787826375.464059:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346363392 00000100:00100000:2.0:1787826375.464079:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11238 00000100:00100000:2.0:1787826375.464087:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3653:x1874663346363648:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826375.464099:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826375.464144:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4c972 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c96b expref: 16 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.464157:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826375.464161:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826375.464200:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3653:x1874663346363648:12345-0@lo:103:ldlm_bl.0 Request processed in 113us (1059us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.464221:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11238 00000100:00100000:2.0:1787826375.464238:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11239 00000100:00100000:2.0:1787826375.464246:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+14:3653:x1874663346363520:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826375.464253:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826375.464264:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c98e lrc: 0/0,0 mode: --/EX res: [0x240000403:0x1e5e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c987 expref: 14 pid: 125377 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.464275:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f794b57a200) 00010000:00010000:2.0:1787826375.464284:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826375.464308:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346363520:12345-0@lo:103:ldlm_bl.0 Request processed in 63us (1107us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.464313:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11239 00000100:00100000:2.0:1787826375.464326:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11240 00000100:00100000:2.0:1787826375.464332:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3654:x1874663346363392:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787826375.464340:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826375.464350:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9aa lrc: 0/0,0 mode: --/EX res: [0x200000401:0x19b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c9a3 expref: 12 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826375.464360:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x0 (ffff9f784abfe500) 00010000:00010000:2.0:1787826375.464368:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826375.464393:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3654:x1874663346363392:12345-0@lo:103:ldlm_bl.0 Request processed in 61us (655us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.464399:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11240 00000100:00100000:2.0:1787826375.464438:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346363520:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787826375.464448:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346363648:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787826375.464582:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346363392:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787826375.857591:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826375.914785:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826375.914836:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826375.914889:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826375.914921:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826375.914952:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826375.914982:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332704384 00000100:00100000:0.0:1787826375.914991:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332704384 00000100:00100000:0.0:1787826375.915055:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332704000 00000100:00100000:0.0:1787826375.915059:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332704000 00000100:00100000:0.0:1787826375.915079:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332704256 00000100:00100000:0.0:1787826375.915081:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332704256 00000100:00100000:0.0:1787826375.915134:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8899 00000100:00100000:0.0:1787826375.915147:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332704384:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826375.915231:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332704384:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 86us (340us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826375.915237:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8899 00000100:00100000:1.0:1787826375.915357:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8900 00000100:00100000:1.0:1787826375.915369:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332704000:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826375.915400:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332704000:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 36us (479us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.915406:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8900 00000100:00100000:1.0:1787826375.915425:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8901 00000100:00100000:1.0:1787826375.915431:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332704256:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826375.915453:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2395:x1874663332704256:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826375.915459:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8901 00000100:00080000:1.0:1787826375.915767:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826375.915771:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826375.916069:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332704128 00000100:00100000:3.0:1787826375.916077:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332704128 00000100:00100000:3.0:1787826375.916174:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14402 00000100:00100000:3.0:1787826375.916184:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332704128:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826375.916239:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332704128:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 56us (1402us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826375.916257:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14402 00000100:00080000:1.0:1787826375.916767:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826375.917304:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332703872 00000100:00100000:2.0:1787826375.917313:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332703872 00000100:00100000:2.0:1787826375.917370:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2246 00000100:00100000:2.0:1787826375.917381:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332703872:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826375.917445:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332703872:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 66us (2671us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826375.917455:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2246 00010000:00010000:3.0:1787826375.997922:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c980 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.998301:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c980 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.998357:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c980 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.998365:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826375.998427:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:3.0:1787826375.998562:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7848ecf800/0xe21309aa81f4c979 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c980 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.998588:0:7460:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826375.998597:0:7460:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.998615:0:7460:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826375.998621:0:7460:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826375.999454:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346363904:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826375.999594:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826375.999873:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346363776:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826375.999899:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787826375.999973:0:7460:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9f7845d43800/0xe21309aa81f4c93a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xe21309aa81f4c941 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787826375.999996:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985503 is committed 00000004:00080000:3.0:1787826376.000027:0:7460:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 371 committed 00000020:00080000:3.0:1787826376.000054:0:7460:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f78513b3000 refcount 2 committed 0 result 0 batchid 3450 00000020:00080000:3.0:1787826376.000058:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abffd00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f78513b3840 00000020:00080000:3.0:1787826376.000062:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1555 00000020:00080000:3.0:1787826376.000064:0:7460:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d007600 00000020:00080000:3.0:1787826376.000067:0:7460:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1236 00000001:00080000:3.0:1787826376.000078:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985497 is committed 00000020:00080000:3.0:1787826376.000090:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3449 batchid updated 00000020:00080000:3.0:1787826376.000131:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3450 committed batchid 3449 00000004:00080000:3.0:1787826376.000256:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d02d600 with version 6903 00000020:00080000:3.0:1787826376.000267:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3446 cancel update log [0x200000401:0x3:0x0].1550: rc = 0 00000020:00080000:3.0:1787826376.000306:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3446 cancel update log [0x240000400:0x3:0x0].1232: rc = 0 00000004:00080000:3.0:1787826376.000345:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d02d6c0 with version 6904 00000020:00080000:3.0:1787826376.000348:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3447 cancel update log [0x200000401:0x3:0x0].1551: rc = 0 00000020:00080000:3.0:1787826376.000365:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3447 cancel update log [0x240000400:0x3:0x0].1233: rc = 0 00000004:00080000:3.0:1787826376.000384:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f784d02d3c0 with version 6905 00000020:00080000:3.0:1787826376.000387:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3448 cancel update log [0x200000401:0x3:0x0].1552: rc = 0 00000020:00080000:3.0:1787826376.000407:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3448 cancel update log [0x240000400:0x3:0x0].1234: rc = 0 00000004:00080000:3.0:1787826376.000423:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7977cfc600 with version 6906 00000020:00080000:3.0:1787826376.000425:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3449 cancel update log [0x200000401:0x3:0x0].1554: rc = 0 00000020:00080000:3.0:1787826376.000438:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3449 cancel update log [0x240000400:0x3:0x0].1235: rc = 0 00000020:00080000:3.0:1787826376.000445:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3449 00000004:00080000:3.0:1787826376.000457:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6903 rpc_version 6903 00000020:00100000:3.0:1787826376.000663:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787826376.000679:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f78508c3b80 x1874663346364160/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826376.000699:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c3b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346364160:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826376.000739:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826376.000759:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483623c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826376.000852:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826376.000864:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346364288:0@lo:6:osp-syn-0-1.0 00000100:00100000:3.0:1787826376.000892:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826376.000924:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346364288 00000100:00100000:3.0:1787826376.000942:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346364288 00000100:00100000:3.0:1787826376.001011:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30900 00000100:00100000:3.0:1787826376.001020:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346364288:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:3.0:1787826376.001052:0:24218:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1233 count 1 00010000:00010000:3.0:1787826376.001076:0:24218:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.001098:0:24218:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 7 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.001128:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x4d1:0x0].0x0 since=8s v=(0 0) 00010000:00010000:3.0:1787826376.001132:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787826376.001137:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.001165:0:24218:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 7 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.001179:0:24218:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 7 pid: 35097 timeout: 8288 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826376.001191:0:24218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b759180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:24218:1874663346364416:192.168.203.56@tcp:104: 00000100:00100000:3.0:1787826376.001266:0:24218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848362000 going to sleep for 11 seconds 00000100:00080000:3.0:1787826376.001278:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826376.004324:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346364160 00000100:00100000:3.0:1787826376.004338:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346364160 00000100:00100000:3.0:1787826376.004399:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24230 00000100:00100000:3.0:1787826376.004409:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7471:x1874663346364160:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826376.005166:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346364160:12345-0@lo:1000:osp_up0-1.0 Request processed in 756us (4423us total) trans 8589957505 rc 0/0 00000100:00100000:3.0:1787826376.005176:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24230 00000100:00100000:1.0:1787826376.005877:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c3b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346364160:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826376.005893:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6904 rpc_version 6904 00000020:00100000:1.0:1787826376.005919:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826376.005926:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7844294a80 x1874663346364672/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826376.005949:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844294a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346364672:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826376.006002:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826376.006046:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826376.006651:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346364672 00000100:00100000:3.0:1787826376.006659:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346364672 00000100:00100000:3.0:1787826376.007256:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24231 00000100:00100000:3.0:1787826376.007270:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7471:x1874663346364672:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826376.007455:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346364672:12345-0@lo:1000:osp_up0-1.0 Request processed in 188us (1452us total) trans 8589957506 rc 0/0 00000100:00100000:3.0:1787826376.007463:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24231 00000100:00100000:1.0:1787826376.008322:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844294a80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346364672:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826376.008337:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6905 rpc_version 6905 00000020:00100000:1.0:1787826376.008361:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826376.008369:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7844295c00 x1874663346364928/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826376.008389:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7844295c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346364928:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826376.008435:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826376.008479:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826376.008503:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346364928 00000100:00100000:1.0:1787826376.008508:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346364928 00000100:00100000:1.0:1787826376.008550:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24232 00000100:00100000:1.0:1787826376.008558:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:7471:x1874663346364928:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826376.008693:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346364928:12345-0@lo:1000:osp_up0-1.0 Request processed in 135us (258us total) trans 8589957507 rc 0/0 00000100:00100000:1.0:1787826376.008701:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24232 00000100:00100000:1.0:1787826376.008736:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7844295c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346364928:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826376.008744:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6906 rpc_version 6906 00000020:00100000:1.0:1787826376.008756:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826376.008760:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784b307b80 x1874663346365184/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826376.008774:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346365184:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826376.008801:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826376.008817:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848334900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826376.008829:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346365184 00000100:00100000:1.0:1787826376.008832:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346365184 00000100:00100000:1.0:1787826376.008857:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24233 00000100:00100000:1.0:1787826376.008863:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+14:7471:x1874663346365184:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826376.008939:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+15:7471:x1874663346365184:12345-0@lo:1000:osp_up0-1.0 Request processed in 76us (138us total) trans 8589957508 rc 0/0 00000100:00100000:1.0:1787826376.008949:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24233 00000100:00100000:1.0:1787826376.008980:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346365184:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826376.011021:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346363904 00000100:00100000:2.0:1787826376.014285:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346363904 00000100:00100000:2.0:1787826376.014384:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346363776 00000100:00100000:2.0:1787826376.014388:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346363776 00000100:00100000:2.0:1787826376.014422:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11241 00000100:00100000:2.0:1787826376.014434:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3655:x1874663346363904:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826376.014451:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826376.014480:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4c941 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19b:0x0].0x5a734ce9 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c93a expref: 16 pid: 8152 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826376.014501:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19b:0x0].0x5a734ce9 (ffff9f794b57a700) 00010000:00010000:2.0:1787826376.014514:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826376.014592:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3655:x1874663346363904:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 161us (14998us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826376.014600:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11241 00000100:00100000:2.0:1787826376.014617:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11242 00000100:00100000:2.0:1787826376.014623:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+14:3655:x1874663346363776:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787826376.014631:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826376.014641:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4c980 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5908076b bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xe21309aa81f4c979 expref: 14 pid: 6532 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826376.014652:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5908076b (ffff9f7967ac1700) 00010000:00010000:2.0:1787826376.014660:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826376.014696:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3655:x1874663346363776:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 73us (14796us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826376.014701:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11242 00000100:00100000:0.0:1787826376.015549:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346363904:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787826376.015563:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346363776:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787826376.022026:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826376.022143:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332704512 00000100:00100000:2.0:1787826376.022152:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332704512 00010000:00010000:2.0:1787826376.022197:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 8 pid: 35097 timeout: 8288 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826376.022243:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22091 00000100:00100000:2.0:1787826376.022253:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c83450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332704512:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826376.022268:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826376.022274:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 8 pid: 35097 timeout: 8288 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826376.022289:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 8 pid: 35097 timeout: 8288 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826376.022316:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826376.022320:0:17812:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4d1:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787826376.022327:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826376.022337:0:17812:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826376.022358:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826376.022419:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c83450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332704512:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 167us (401us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826376.022426:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22091 00010000:00010000:3.0:1787826376.023991:0:24218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bee640c/0xe21309aa81f4c75e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801c2f expref: 7 pid: 35097 timeout: 8288 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826376.024014:0:24218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b759180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:24218:1874663346364416:192.168.203.56@tcp:104: 00010000:00010000:3.0:1787826376.024055:0:24218:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.024066:0:24218:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.024073:0:24218:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.024085:0:24218:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.024098:0:24218:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826376.024326:0:24218:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826376.024635:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826376.024643:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826376.024661:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:0.0:1787826376.024665:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826376.024683:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826376.024707:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826376.024714:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f94a9c48/0xe21309aa81f4c9b1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4d1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826376.024735:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826376.024737:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826376.024739:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826376.024838:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+6:3653:x1874663346364288:12345-0@lo:6:osp-syn-0-1.0 Request processed in 23816us (23941us total) trans 4294973866 rc 0/0 00000100:00100000:3.0:1787826376.024847:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30900 00000004:00080000:1.0:1787826376.028729:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784b75ad80/2, rc 0, transno 4294973866 00000100:00100000:1.0:1787826376.028746:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346364288:0@lo:6:osp-syn-0-1.0 00000100:00100000:2.0:1787826376.686626:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826376.686744:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826376.686768:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346365696:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.686857:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826376.686928:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346365696 00000100:00100000:3.0:1787826376.686935:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346365696 00000100:00100000:3.0:1787826376.686994:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14403 00000100:00100000:3.0:1787826376.687006:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346365696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.687247:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b306300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346365312:0@lo:400:kworker.0 00000100:00100000:0.0:1787826376.687282:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346365440:0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.687291:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826376.687327:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826376.687347:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346365312 00000100:00100000:2.0:1787826376.687351:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346365312 00000100:00100000:0.0:1787826376.687356:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826376.687364:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346365568:0@lo:400:kworker.0 00000100:00100000:0.0:1787826376.687390:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826376.687393:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2247 00000100:00100000:2.0:1787826376.687403:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784331f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346365312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826376.687440:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346365440 00000100:00100000:2.0:1787826376.687445:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784331f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346365312:12345-0@lo:400:kworker.0 Request processed in 43us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826376.687446:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346365440 00000100:00100000:2.0:1787826376.687456:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2247 00000100:00100000:0.0:1787826376.687483:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346365568 00000100:00100000:0.0:1787826376.687496:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346365568 00000100:00100000:2.0:1787826376.687510:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b306300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346365312:0@lo:400:kworker.0 00000100:00100000:0.0:1787826376.687528:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14404 00000100:00100000:0.0:1787826376.687537:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346365440:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.687564:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14405 00000100:00100000:2.0:1787826376.687574:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346365568:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826376.687580:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346365440:12345-0@lo:400:kworker.0 Request processed in 43us (252us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826376.687588:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14404 00000100:00100000:2.0:1787826376.687611:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346365568:12345-0@lo:400:kworker.0 Request processed in 39us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826376.687618:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14405 00000100:00100000:0.0:1787826376.687638:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346365568:0@lo:400:kworker.0 00000100:00080000:2.0:1787826376.687642:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826376.687648:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826376.687654:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346365440:0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.687678:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346365696:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.687723:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346365696:12345-0@lo:400:kworker.0 Request processed in 66us (211us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826376.687732:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14403 00000100:00100000:3.0:1787826376.690508:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826376.690612:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346366080:0@lo:400:kworker.0 00000100:00100000:0.0:1787826376.690707:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826376.690755:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787826376.690761:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346365952:0@lo:400:kworker.0 00000100:00100000:0.0:1787826376.690788:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826376.690834:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346366080 00000100:00100000:0.0:1787826376.690842:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346366080 00000100:00100000:0.0:1787826376.690914:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8902 00000100:00100000:0.0:1787826376.690924:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346366080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826376.690974:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346366080:12345-0@lo:400:kworker.0 Request processed in 53us (268us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826376.690982:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8902 00000100:00100000:0.0:1787826376.691010:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346365952 00000100:00100000:0.0:1787826376.691014:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346365952 00000100:00100000:0.0:1787826376.691057:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14406 00000100:00100000:0.0:1787826376.691064:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346365952:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.691444:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826376.691458:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346366208:0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.691510:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1787826376.691573:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826376.691579:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826376.691624:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346366080:0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.691656:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346366208 00000100:00100000:2.0:1787826376.691660:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346366208 00000100:00100000:2.0:1787826376.691693:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24234 00000100:00100000:2.0:1787826376.691701:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346366208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.691750:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346366208:12345-0@lo:400:kworker.0 Request processed in 50us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826376.691757:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24234 00000100:00100000:2.0:1787826376.691786:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985497 gen 2 00000004:00080000:2.0:1787826376.691793:0:3653:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985496 committed transno 4294985497 00000004:00080000:2.0:1787826376.691819:0:3653:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985497 committed transno 4294985497 00000020:00080000:2.0:1787826376.691826:0:3653:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff9f7846318e40 refcount 2 committed 0 result 0 batchid 4294979091 00000020:00080000:2.0:1787826376.691831:0:3653:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784a925f00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7846318240 00000020:00080000:2.0:1787826376.691835:0:3653:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000404:0x2:0x0].948 00000020:00080000:2.0:1787826376.691838:0:3653:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784a925200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f78492e3000 00000020:00080000:2.0:1787826376.691841:0:3653:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000bd0:0x2:0x0].945 00000100:00100000:2.0:1787826376.691864:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346366208:0@lo:400:kworker.0 00000020:00080000:2.0:1787826376.691896:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 4294979091 committed batchid 4294979090 00000020:00080000:2.0:1787826376.691972:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979090 00000100:00100000:0.0:1787826376.692388:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346365952:12345-0@lo:400:kworker.0 Request processed in 1323us (1597us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826376.692397:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14406 00000100:00100000:2.0:1787826376.692925:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346365952:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.693254:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346365824:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.693307:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826376.693615:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826376.693625:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346366336:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.693671:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826376.693712:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346366464:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.693737:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826376.693757:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826376.693762:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346366592:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.693785:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826376.693838:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346366336 00000100:00100000:1.0:1787826376.693844:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346366336 00000100:00100000:1.0:1787826376.693904:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346366464 00000100:00100000:1.0:1787826376.693908:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346366464 00000100:00100000:1.0:1787826376.693928:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8903 00000100:00100000:1.0:1787826376.693937:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346366336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826376.693987:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346366336:12345-0@lo:400:kworker.0 Request processed in 51us (315us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826376.693993:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8903 00000100:00100000:1.0:1787826376.694008:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8904 00000100:00100000:1.0:1787826376.694014:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346366464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826376.694054:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346366464:12345-0@lo:400:kworker.0 Request processed in 40us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826376.694060:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8904 00000100:00080000:0.0:1787826376.694838:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826376.695250:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346366464:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.695297:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346366336:0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.695819:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346365824 00000100:00100000:2.0:1787826376.695828:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346365824 00000100:00100000:2.0:1787826376.695905:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346366592 00000100:00100000:2.0:1787826376.695908:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346366592 00000100:00100000:2.0:1787826376.695929:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14407 00000100:00100000:2.0:1787826376.695938:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346365824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.696015:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a91f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346365824:12345-0@lo:400:kworker.0 Request processed in 78us (2708us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826376.696622:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14408 00000100:00100000:3.0:1787826376.696635:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346366592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.696700:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346366592:12345-0@lo:400:kworker.0 Request processed in 70us (2913us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826376.696708:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14408 00000100:00100000:3.0:1787826376.696765:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346366592:0@lo:400:kworker.0 00000100:00100000:3.0:1787826376.696813:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8be680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346365824:0@lo:400:kworker.0 00000100:00100000:2.0:1787826376.697363:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14407 00000100:00080000:0.0:1787826376.697452:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787826376.876967:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1787826377.012281:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973866 is committed 00000001:00080000:2.0:1787826377.012299:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973865 is committed 00000100:00100000:2.0:1787826377.198128:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826377.198155:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346366720:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826377.198249:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826377.198440:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346366720 00000100:00100000:0.0:1787826377.198449:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346366720 00000100:00100000:0.0:1787826377.198513:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6275 00000100:00100000:0.0:1787826377.198523:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346366720:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826377.198553:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826377.198612:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346366720:12345-0@lo:13:osp-pre-1-1.0 Request processed in 90us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826377.198619:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6275 00000100:00100000:1.0:1787826377.198766:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346366720:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826377.709638:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346366848:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826377.709718:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826377.709784:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346366976:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826377.709814:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826377.709851:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346366976 00000100:00100000:3.0:1787826377.709858:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346366976 00000100:00100000:3.0:1787826377.709911:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6276 00000100:00100000:3.0:1787826377.709922:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346366976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787826377.709946:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826377.709990:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346366976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 71us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826377.710000:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6276 00000100:00080000:3.0:1787826377.710021:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787826377.710056:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346366976:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826377.710612:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346366848 00000100:00100000:1.0:1787826377.710619:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346366848 00000100:00100000:1.0:1787826377.710657:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24235 00000100:00100000:1.0:1787826377.710665:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346366848:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826377.710698:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:1.0:1787826377.710751:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:3653:x1874663346366848:12345-0@lo:41:osp-pre-0-1.0 Request processed in 87us (1033us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826377.710758:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24235 00000100:00100000:3.0:1787826377.712667:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346366848:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826377.712693:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826377.712703:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346367232:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826377.712757:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826377.712797:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346367104:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826377.712824:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826377.713686:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346367232 00000100:00100000:3.0:1787826377.713695:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346367232 00000100:00100000:3.0:1787826377.713737:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346367104 00000100:00100000:3.0:1787826377.713740:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346367104 00000100:00100000:3.0:1787826377.713765:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6277 00000100:00100000:3.0:1787826377.713775:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346367232:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787826377.713803:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:3.0:1787826377.713861:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346367232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (1103us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826377.713867:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6277 00000100:00100000:3.0:1787826377.713883:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6278 00000100:00100000:3.0:1787826377.713889:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7975895050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346367104:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826377.713898:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:3.0:1787826377.713924:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7975895050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346367104:12345-0@lo:13:osp-pre-0-1.0 Request processed in 36us (1099us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826377.713930:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6278 00000100:00080000:3.0:1787826377.716556:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826377.716637:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346367232:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826377.716681:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294973866 gen 2 00000004:00080000:3.0:1787826377.716687:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784b75ad80, transno 4294973866 00000100:00100000:3.0:1787826377.716708:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346367104:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826377.725148:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826377.725256:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346367360:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826377.725338:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826377.742568:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346367360 00000100:00100000:2.0:1787826377.742582:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346367360 00000100:00100000:2.0:1787826377.742636:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24236 00000100:00100000:2.0:1787826377.742645:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346367360:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826377.742682:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:2.0:1787826377.742775:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346367360:12345-0@lo:41:osp-pre-1-0.0 Request processed in 131us (17438us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826377.742782:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24236 00000100:00100000:2.0:1787826377.742845:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346367360:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1787826377.900609:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826378.924833:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826379.950349:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:2.0:1787826380.789348:0:30894:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 4294979091 batchid updated 00000001:00080000:2.0:1787826380.789381:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957508 is committed 00000020:00080000:3.0:1787826380.789445:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979091 00000004:00080000:3.0:1787826380.789510:0:30923:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffff9f78483626c0 with version 1844 00000020:00080000:3.0:1787826380.789528:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 4294979091 cancel update log [0x240000404:0x2:0x0].948: rc = 0 00000020:00080000:3.0:1787826380.789611:0:30923:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 4294979091 cancel update log [0x200000bd0:0x2:0x0].945: rc = 0 00000020:00080000:3.0:1787826380.789619:0:30923:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 4294979091 00000004:00080000:2.0:1787826380.789655:0:30909:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f797cf39cc0 version 1844 rpc_version 1844 00000020:00100000:2.0:1787826380.789691:0:30909:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787826380.789702:0:30909:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f784d9ded80 x1874663346367616/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826380.789729:0:30909:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ded80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346367616:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787826380.789810:0:30909:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826380.789883:0:30909:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848d5e0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826380.789942:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346367616 00000100:00100000:3.0:1787826380.789951:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346367616 00000100:00100000:3.0:1787826380.790024:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24237 00000100:00100000:3.0:1787826380.790035:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:30909:x1874663346367616:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787826380.790231:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:30909:x1874663346367616:12345-0@lo:1000:osp_up1-0.0 Request processed in 196us (420us total) trans 4294985504 rc 0/0 00000100:00100000:3.0:1787826380.790239:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24237 00000100:00100000:2.0:1787826380.790397:0:30909:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ded80 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:30909:1874663346367616:0@lo:1000:osp_up1-0.0 00000001:00080000:1.0:1787826380.799016:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976175 is committed 00010000:00010000:3.0:1787826380.975555:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826381.024946:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.024995:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.025036:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.025066:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.025089:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.025139:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.025178:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.025205:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.025315:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826381.025347:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332705152 00000100:00100000:1.0:1787826381.025357:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332705152 00000100:00100000:1.0:1787826381.025421:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332705664 00000100:00100000:1.0:1787826381.025424:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332705664 00000100:00100000:1.0:1787826381.025452:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332705024 00000100:00100000:1.0:1787826381.025455:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332705024 00000100:00100000:1.0:1787826381.025475:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332705536 00000100:00100000:1.0:1787826381.025478:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332705536 00000100:00100000:1.0:1787826381.025503:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8905 00000100:00100000:1.0:1787826381.025513:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332705152:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826381.025583:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332705152:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 71us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.025591:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8905 00000100:00100000:1.0:1787826381.025609:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8906 00000100:00100000:1.0:1787826381.025616:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332705664:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826381.025648:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332705664:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 33us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.025655:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8906 00000100:00100000:1.0:1787826381.025670:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8907 00000100:00100000:1.0:1787826381.025678:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332705024:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826381.025704:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332705024:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (526us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.025711:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8907 00000100:00100000:1.0:1787826381.025732:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8908 00000100:00100000:1.0:1787826381.025740:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332705536:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826381.025751:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332704896 00000100:00100000:0.0:1787826381.025757:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332704896 00000100:00100000:1.0:1787826381.025767:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332705536:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (452us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.025776:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8908 00000100:00100000:1.0:1787826381.025804:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332705408 00000100:00100000:1.0:1787826381.025809:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332705408 00000100:00100000:1.0:1787826381.025844:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332704768 00000100:00100000:1.0:1787826381.025848:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332704768 00000100:00100000:1.0:1787826381.025872:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332705280 00000100:00100000:1.0:1787826381.025875:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332705280 00000100:00100000:1.0:1787826381.025901:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14409 00000100:00100000:1.0:1787826381.025910:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332705408:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826381.025939:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332705408:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 32us (873us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.025946:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14409 00000100:00100000:1.0:1787826381.025959:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14410 00000100:00100000:1.0:1787826381.025966:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332704768:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826381.025989:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332704768:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (850us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.025995:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14410 00000100:00100000:1.0:1787826381.026006:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14411 00000100:00100000:1.0:1787826381.026012:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332705280:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826381.026075:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14412 00000100:00100000:0.0:1787826381.026085:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332704896:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826381.026135:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332704896:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 52us (1139us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826381.026141:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14412 00000100:00100000:3.0:1787826381.026375:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332704640 00000100:00100000:3.0:1787826381.026381:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332704640 00000100:00100000:3.0:1787826381.026411:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2248 00000100:00100000:3.0:1787826381.026418:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332704640:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826381.026449:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332704640:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 31us (1515us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826381.026456:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2248 00000100:00100000:1.0:1787826381.027127:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332705280:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 1110us (1917us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.027135:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14411 00000100:00080000:1.0:1787826381.027190:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787826381.027195:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787826381.027199:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826381.027202:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826381.806939:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346367872:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807014:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807071:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346368384:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807098:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807173:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851835180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346368896:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807197:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807224:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1787826381.807230:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346367744:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807257:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807278:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346368256:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807300:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807313:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851836a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346368768:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807330:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807354:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346368000:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807375:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807387:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967912680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346368512:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807402:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807416:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787826381.807421:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7848cfd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346368128:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807439:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807456:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851836680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346368640:0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807474:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826381.807502:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346367872 00000100:00100000:2.0:1787826381.807510:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346367872 00000100:00100000:2.0:1787826381.807567:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346368896 00000100:00100000:2.0:1787826381.807571:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346368896 00000100:00100000:2.0:1787826381.807591:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24238 00000100:00100000:2.0:1787826381.807600:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346367872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.807653:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346367872:12345-0@lo:400:kworker.0 Request processed in 54us (640us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826381.807659:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24238 00000100:00100000:0.0:1787826381.807872:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346368256 00000100:00100000:0.0:1787826381.807877:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346368256 00000100:00100000:0.0:1787826381.807914:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346368000 00000100:00100000:0.0:1787826381.807917:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346368000 00000100:00100000:0.0:1787826381.807935:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346368512 00000100:00100000:0.0:1787826381.807938:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346368512 00000100:00100000:0.0:1787826381.807956:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346368128 00000100:00100000:0.0:1787826381.807959:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346368128 00000100:00100000:0.0:1787826381.807973:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346368640 00000100:00100000:0.0:1787826381.807975:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346368640 00000100:00100000:0.0:1787826381.807992:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14413 00000100:00100000:0.0:1787826381.807999:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346368256:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826381.808038:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346368256:12345-0@lo:400:kworker.0 Request processed in 40us (736us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826381.808044:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14413 00000100:00100000:0.0:1787826381.808059:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14414 00000100:00100000:0.0:1787826381.808065:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346368000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826381.808099:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b305f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346368000:12345-0@lo:400:kworker.0 Request processed in 34us (722us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826381.808132:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14414 00000100:00100000:0.0:1787826381.808145:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14415 00000100:00100000:0.0:1787826381.808152:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346368512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826381.808188:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346368512:12345-0@lo:400:kworker.0 Request processed in 37us (784us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826381.808194:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14415 00000100:00100000:0.0:1787826381.808207:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14416 00000100:00100000:0.0:1787826381.808213:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346368128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826381.808239:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b306300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346368128:12345-0@lo:400:kworker.0 Request processed in 27us (798us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826381.808244:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14416 00000100:00100000:0.0:1787826381.808254:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14417 00000100:00100000:0.0:1787826381.808259:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346368640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826381.808279:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346368640:12345-0@lo:400:kworker.0 Request processed in 20us (804us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826381.808284:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14417 00000100:00100000:2.0:1787826381.808386:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24239 00000100:00100000:2.0:1787826381.808393:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346368896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.808423:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346368896:12345-0@lo:400:kworker.0 Request processed in 32us (1226us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826381.808429:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24239 00000100:00100000:2.0:1787826381.808451:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346367744 00000100:00100000:2.0:1787826381.808455:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346367744 00000100:00100000:2.0:1787826381.808491:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2249 00000100:00100000:2.0:1787826381.808498:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784331d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346367744:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.808528:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784331d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346367744:12345-0@lo:400:kworker.0 Request processed in 31us (1270us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826381.808533:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2249 00000100:00100000:2.0:1787826381.808578:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346368384 00000100:00100000:2.0:1787826381.808582:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346368384 00000100:00100000:2.0:1787826381.808607:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346368768 00000100:00100000:2.0:1787826381.808610:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346368768 00000100:00100000:2.0:1787826381.808627:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8909 00000100:00100000:2.0:1787826381.808633:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346368384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.808663:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346368384:12345-0@lo:400:kworker.0 Request processed in 30us (1564us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826381.808814:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346369152:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.808850:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826381.808873:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826381.808879:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346369024:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.808902:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826381.808986:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346367744:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809002:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346368256:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809015:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957508 gen 2 00000004:00080000:3.0:1787826381.809020:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957503 committed transno 8589957508 00000020:00080000:3.0:1787826381.809026:0:3655:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7850705a80 refcount 2 committed 0 result 0 batchid 3451 00000020:00080000:3.0:1787826381.809030:0:3655:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784a925b00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f784549ae40 00000020:00080000:3.0:1787826381.809033:0:3655:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1556 00000020:00080000:3.0:1787826381.809036:0:3655:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f784abff200 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784d007400 00000020:00080000:3.0:1787826381.809039:0:3655:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1237 00000004:00080000:3.0:1787826381.809061:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957504 committed transno 8589957508 00000020:00080000:3.0:1787826381.809065:0:3655:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9f7848d5ee40 refcount 2 committed 0 result 0 batchid 3452 00000020:00080000:3.0:1787826381.809068:0:3655:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9f7848d5e3c0 00000020:00080000:3.0:1787826381.809071:0:3655:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].1557 00000020:00080000:3.0:1787826381.809073:0:3655:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9f78508d1c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9f784a191a00 00000020:00080000:3.0:1787826381.809075:0:3655:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].1238 00000004:00080000:3.0:1787826381.809086:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957505 committed transno 8589957508 00000004:00080000:3.0:1787826381.809100:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957506 committed transno 8589957508 00000004:00080000:3.0:1787826381.809249:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957507 committed transno 8589957508 00000004:00080000:3.0:1787826381.809259:0:3655:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957508 committed transno 8589957508 00000100:00100000:3.0:1787826381.809274:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346367872:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809288:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977f29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346368384:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809298:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851835180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346368896:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809319:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826381.809326:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346369280:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809353:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:3.0:1787826381.809375:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3452 committed batchid 3449 00000020:00080000:3.0:1787826381.809427:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3449 00000100:00100000:3.0:1787826381.809456:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346368128:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809468:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851836680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346368640:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809478:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7848cfed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346368000:0@lo:400:kworker.0 00000100:00100000:3.0:1787826381.809489:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967912680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346368512:0@lo:400:kworker.0 00000100:00100000:1.0:1787826381.810065:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346369152 00000100:00100000:1.0:1787826381.810072:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346369152 00000100:00100000:2.0:1787826381.810617:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8909 00000100:00100000:2.0:1787826381.810643:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346369024 00000100:00100000:2.0:1787826381.810647:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346369024 00000100:00100000:1.0:1787826381.810857:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8910 00000100:00100000:1.0:1787826381.810869:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346368768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826381.810954:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346368768:12345-0@lo:400:kworker.0 Request processed in 88us (3622us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.810960:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8910 00000100:00100000:1.0:1787826381.810974:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8911 00000100:00100000:1.0:1787826381.810981:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346369152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826381.811015:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346369152:12345-0@lo:400:kworker.0 Request processed in 36us (2165us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.811021:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8911 00000100:00100000:1.0:1787826381.811033:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8912 00000100:00100000:1.0:1787826381.811040:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b661880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346369024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826381.811069:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b661880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346369024:12345-0@lo:400:kworker.0 Request processed in 31us (2167us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826381.811075:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8912 00000100:00100000:2.0:1787826381.811448:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346369280 00000100:00100000:2.0:1787826381.811454:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346369280 00000100:00100000:2.0:1787826381.811489:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14418 00000100:00100000:2.0:1787826381.811497:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346369280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826381.811540:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b660000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346369280:12345-0@lo:400:kworker.0 Request processed in 45us (2186us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826381.811546:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14418 00000100:00100000:1.0:1787826381.811747:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346369024:0@lo:400:kworker.0 00000100:00100000:1.0:1787826381.811761:0:3653:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976175 gen 2 00000004:00080000:1.0:1787826381.811765:0:3653:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784d9fed80, transno 4294976175 00000100:00100000:1.0:1787826381.811777:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346369152:0@lo:400:kworker.0 00000100:00100000:1.0:1787826381.811919:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346369280:0@lo:400:kworker.0 00000100:00100000:1.0:1787826381.811933:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851836a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346368768:0@lo:400:kworker.0 00000100:00080000:1.0:1787826381.811962:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826381.811967:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787826381.811971:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787826381.811973:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000020:00080000:3.0:1787826381.818014:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3452 batchid updated 00000001:00080000:3.0:1787826381.818031:0:7460:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294985504 is committed 00000020:00080000:3.0:1787826381.818042:0:7460:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 3450 batchid updated 00000020:00080000:0.0:1787826381.818523:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3452 00000004:00080000:0.0:1787826381.818592:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852f2a540 with version 6907 00000020:00080000:0.0:1787826381.818613:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3450 cancel update log [0x200000401:0x3:0x0].1555: rc = 0 00000020:00080000:0.0:1787826381.818697:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3450 cancel update log [0x240000400:0x3:0x0].1236: rc = 0 00000004:00080000:0.0:1787826381.818725:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852f2a300 with version 6908 00000020:00080000:0.0:1787826381.818728:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3451 cancel update log [0x200000401:0x3:0x0].1556: rc = 0 00000020:00080000:0.0:1787826381.818746:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3451 cancel update log [0x240000400:0x3:0x0].1237: rc = 0 00000004:00080000:0.0:1787826381.818767:0:7481:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9f7852f2a780 with version 6909 00000020:00080000:0.0:1787826381.818770:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3452 cancel update log [0x200000401:0x3:0x0].1557: rc = 0 00000020:00080000:0.0:1787826381.818788:0:7481:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3452 cancel update log [0x240000400:0x3:0x0].1238: rc = 0 00000020:00080000:0.0:1787826381.818795:0:7481:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3452 00000004:00080000:1.0:1787826381.819225:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6907 rpc_version 6907 00000020:00100000:1.0:1787826381.819261:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826381.819273:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7850771880 x1874663346369536/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826381.819298:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346369536:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826381.819375:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826381.819423:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483346c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826381.821238:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346369536 00000100:00100000:2.0:1787826381.821254:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346369536 00000100:00100000:2.0:1787826381.821340:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24240 00000100:00100000:2.0:1787826381.821350:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7471:x1874663346369536:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787826381.821581:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346369536:12345-0@lo:1000:osp_up0-1.0 Request processed in 231us (2204us total) trans 8589957509 rc 0/0 00000100:00100000:2.0:1787826381.821588:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24240 00000100:00100000:1.0:1787826381.822238:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346369536:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826381.822254:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6908 rpc_version 6908 00000020:00100000:1.0:1787826381.822278:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826381.822285:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7850ad4380 x1874663346369792/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826381.822310:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad4380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346369792:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826381.822353:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826381.822385:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483346c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826381.822462:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346369792 00000100:00100000:3.0:1787826381.822469:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346369792 00000100:00100000:3.0:1787826381.822518:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24241 00000100:00100000:3.0:1787826381.822528:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7471:x1874663346369792:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826381.822653:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346369792:12345-0@lo:1000:osp_up0-1.0 Request processed in 125us (297us total) trans 8589957510 rc 0/0 00000100:00100000:3.0:1787826381.822661:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24241 00000100:00100000:1.0:1787826381.822784:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad4380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346369792:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787826381.822792:0:7471:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9f7946752900 version 6909 rpc_version 6909 00000020:00100000:1.0:1787826381.822804:0:7471:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787826381.822809:0:7471:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9f7850ad5180 x1874663346370048/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826381.822823:0:7471:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad5180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346370048:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787826381.822852:0:7471:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826381.822879:0:7471:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78483346c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826381.822926:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346370048 00000100:00100000:3.0:1787826381.822932:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346370048 00000100:00100000:3.0:1787826381.822962:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24242 00000100:00100000:3.0:1787826381.822970:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7471:x1874663346370048:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787826381.823052:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7471:x1874663346370048:12345-0@lo:1000:osp_up0-1.0 Request processed in 82us (198us total) trans 8589957511 rc 0/0 00000100:00100000:3.0:1787826381.823059:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24242 00000100:00100000:1.0:1787826381.823292:0:7471:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad5180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7471:1874663346370048:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787826381.996794:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826382.316994:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346370176:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826382.317079:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826382.317285:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346370176 00000100:00100000:3.0:1787826382.317293:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346370176 00000100:00100000:3.0:1787826382.317352:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6279 00000100:00100000:3.0:1787826382.317362:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346370176:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826382.317391:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826382.318296:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346370176:12345-0@lo:13:osp-pre-1-1.0 Request processed in 922us (1203us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826382.318306:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6279 00000100:00100000:1.0:1787826382.319341:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346370176:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826382.828860:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7967912680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346370304:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826382.828930:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826382.828966:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826382.828971:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851836a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346370432:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826382.829006:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826382.829030:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851836680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346370560:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826382.829050:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826382.829076:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346370304 00000100:00100000:3.0:1787826382.829082:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346370304 00000100:00100000:3.0:1787826382.829135:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346370560 00000100:00100000:3.0:1787826382.829138:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346370560 00000100:00100000:3.0:1787826382.829164:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6280 00000100:00100000:3.0:1787826382.829175:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346370304:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826382.829222:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:3.0:1787826382.829265:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346370304:12345-0@lo:13:osp-pre-0-1.0 Request processed in 92us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826382.829271:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6280 00000100:00100000:3.0:1787826382.829290:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6281 00000100:00100000:3.0:1787826382.829302:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346370560:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787826382.829312:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826382.829341:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346370560:12345-0@lo:13:osp-pre-1-0.0 Request processed in 46us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826382.829347:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6281 00000100:00100000:3.0:1787826382.829365:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346370432 00000100:00100000:3.0:1787826382.829368:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346370432 00000100:00100000:3.0:1787826382.829393:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24243 00000100:00100000:3.0:1787826382.829400:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346370432:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787826382.829567:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:1.0:1787826382.830298:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346370688:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826382.833604:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7967912680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346370304:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826382.833684:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851836680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346370560:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826382.834860:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:3652:x1874663346370432:12345-0@lo:41:osp-pre-0-1.0 Request processed in 5454us (5847us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826382.834873:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24243 00000100:00100000:2.0:1787826382.835047:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346370816:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826382.835413:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826382.848228:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826382.848350:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851836a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346370432:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826382.848405:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346370816 00000100:00100000:2.0:1787826382.848412:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346370816 00000100:00100000:2.0:1787826382.848547:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24244 00000100:00100000:2.0:1787826382.848557:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851836680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346370816:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826382.848592:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:2.0:1787826382.848659:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851836680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346370816:12345-0@lo:41:osp-pre-1-0.0 Request processed in 103us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826382.848666:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24244 00000100:00100000:2.0:1787826382.848701:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294985504 gen 2 00000004:00080000:2.0:1787826382.848707:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294985504 committed transno 4294985504 00000100:00100000:2.0:1787826382.848736:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346370816:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826382.851128:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346370688 00000100:00100000:2.0:1787826382.851142:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346370688 00000100:00100000:2.0:1787826382.851213:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6282 00000100:00100000:2.0:1787826382.851223:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346370688:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826382.851257:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826382.851334:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346370688:12345-0@lo:13:osp-pre-0-0.0 Request processed in 112us (15922us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826382.851342:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6282 00000100:00100000:2.0:1787826382.851394:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847704380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346370688:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787826383.022251:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826383.595217:0:167955:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 06:26:20 (1787826380) 00000100:00100000:3.0:1787826383.692059:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826383.692776:0:125406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332705792 00000100:00100000:2.0:1787826383.692788:0:125406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332705792 00000100:00100000:2.0:1787826383.692873:0:125406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11243 00000100:00100000:2.0:1787826383.692887:0:125406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:15672:x1874663332705792:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826383.692904:0:125406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826383.692932:0:125406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4c8a7 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1dc3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c91 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826383.692960:0:125406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1dc3:0x0].0x0 (ffff9f784abfe300) 00010000:00010000:2.0:1787826383.692973:0:125406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787826383.693033:0:125406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:15672:x1874663332705792:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 150us (984us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826383.693040:0:125406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11243 00000100:00080000:0.0:1787826383.693238:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826383.699492:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826383.699607:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332705920 00000100:00100000:3.0:1787826383.699618:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332705920 00000100:00100000:3.0:1787826383.699691:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 11244 00000100:00100000:3.0:1787826383.699704:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:15673:x1874663332705920:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826383.699719:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826383.699745:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9f7845d42200/0xe21309aa81f4c806 lrc: 0/0,0 mode: --/PR res: [0x240000403:0x1e5e:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801c6e expref: 6 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826383.699767:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1e5e:0x0].0x0 (ffff9f784abfe100) 00010000:00010000:3.0:1787826383.699781:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826383.699838:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:15673:x1874663332705920:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 138us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826383.699845:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 11244 00000100:00080000:0.0:1787826383.700087:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:0.0:1787826384.052175:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826385.068289:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826385.106949:0:168003:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:3.0:1787826385.107000:0:168003:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:1.0:1787826386.095400:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826386.145323:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145370:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145397:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145419:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145464:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145486:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145504:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145523:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145612:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.145655:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332706560 00000100:00100000:2.0:1787826386.145667:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332706560 00000100:00100000:2.0:1787826386.145739:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332707072 00000100:00100000:2.0:1787826386.145742:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332707072 00000100:00100000:2.0:1787826386.145763:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332706432 00000100:00100000:2.0:1787826386.145766:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332706432 00000100:00100000:2.0:1787826386.145785:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332706944 00000100:00100000:2.0:1787826386.145787:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332706944 00000100:00100000:2.0:1787826386.145816:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8913 00000100:00100000:2.0:1787826386.145827:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332706560:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826386.145919:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847704380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332706560:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 93us (548us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.145930:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8913 00000100:00100000:2.0:1787826386.145949:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8914 00000100:00100000:2.0:1787826386.145956:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967912680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332707072:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826386.145979:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967912680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332707072:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (560us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.145985:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8914 00000100:00100000:2.0:1787826386.145996:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8915 00000100:00100000:2.0:1787826386.146003:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332706944:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826386.146026:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332706944:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (415us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.146032:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8915 00000100:00100000:2.0:1787826386.146044:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30901 00000100:00100000:2.0:1787826386.146051:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2392:x1874663332706432:12345-192.168.203.56@tcp:9:kworker.0 00000020:00000080:2.0:1787826386.146075:0:8444:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xe21309aa81f4c50b 00000100:00080000:2.0:1787826386.146085:0:8444:0:(import.c:65:import_set_state_nolock()) ffff9f78435ba000 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1787826386.146181:0:8444:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294973866 00000100:00100000:2.0:1787826386.146338:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844297b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+2:2392:x1874663332706432:12345-192.168.203.56@tcp:9:kworker.0 Request processed in 287us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.146348:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30901 00000100:00100000:2.0:1787826386.146477:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332706176 00000100:00100000:2.0:1787826386.146483:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332706176 00000100:00100000:2.0:1787826386.146520:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332706816 00000100:00100000:2.0:1787826386.146523:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332706816 00000100:00100000:2.0:1787826386.146544:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332706304 00000100:00100000:2.0:1787826386.146547:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332706304 00000100:00100000:2.0:1787826386.146566:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332706688 00000100:00100000:2.0:1787826386.146569:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332706688 00000100:00100000:2.0:1787826386.146591:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14419 00000100:00100000:2.0:1787826386.146598:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851836a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332706176:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826386.146629:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851836a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332706176:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 31us (1323us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.146635:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14419 00000100:00100000:2.0:1787826386.146649:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14420 00000100:00100000:2.0:1787826386.146655:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332706816:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826386.146680:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332706816:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 26us (1283us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.146686:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14420 00000100:00100000:2.0:1787826386.146695:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14421 00000100:00100000:2.0:1787826386.146701:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844295c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332706304:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826386.146722:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844295c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332706304:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (1235us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.146728:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14421 00000100:00100000:2.0:1787826386.146739:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14422 00000100:00100000:2.0:1787826386.146745:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332706688:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826386.146765:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332706688:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 20us (1259us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.146770:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14422 00000100:00100000:2.0:1787826386.146799:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332706048 00000100:00100000:2.0:1787826386.146803:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332706048 00000100:00100000:2.0:1787826386.146832:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2250 00000100:00100000:2.0:1787826386.146839:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332706048:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826386.146860:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332706048:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (1396us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.146867:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2250 00000100:00080000:2.0:1787826386.146913:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826386.146918:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826386.146920:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826386.146923:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787826386.147138:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826386.237276:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826386.237874:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332707200 00000100:00100000:3.0:1787826386.237889:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332707200 00000100:00100000:2.0:1787826386.238071:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68733 00000100:00100000:2.0:1787826386.238087:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:227600:x1874663332707200:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:2.0:1787826386.238145:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826386.238174:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c9b8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cc9 expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826386.238253:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9bf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826386.238272:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826386.238286:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826386.238344:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826386.238347:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826386.238371:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826386.238410:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4c9b8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cc9 expref: 8 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826386.238430:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9bf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cc9 expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826386.238439:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826386.238443:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826386.238447:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490efc00, rc 0) 00000100:00100000:2.0:1787826386.238520:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7844294e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:227600:x1874663332707200:12345-192.168.203.56@tcp:101:touch.0 Request processed in 437us (1259us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826386.238528:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68733 00000100:00100000:2.0:1787826386.252863:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826386.257631:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332707456 00000100:00100000:1.0:1787826386.257652:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332707456 00000100:00100000:1.0:1787826386.257750:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68734 00000100:00100000:1.0:1787826386.257764:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:227600:x1874663332707456:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:1.0:1787826386.257797:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826386.257802:0:35095:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787826386.257839:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4c9bf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cc9 expref: 7 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.257867:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826386.257874:0:35095:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826386.257892:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c9c6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cd0 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.264724:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.264823:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.264838:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.265040:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x508acae2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.265049:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.265059:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787826386.266682:0:35095:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x524:0x0], index 0: 0 00010000:00010000:1.0:1787826386.267069:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c9db lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267087:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c9db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267101:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c9db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267175:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267182:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267188:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267195:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267201:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826386.267207:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267213:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267225:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826386.267229:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826386.267233:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267239:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf000/0xe21309aa81f4c9cd lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826386.267257:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000092554fe9 reply state 000000007412ed9c transno 8589957512 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267268:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267274:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x508acae2 (ffff9f7845cb1a00) 00010000:00010000:1.0:1787826386.267276:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826386.267279:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787826386.267320:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4c9c6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cd0 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267632:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c9db lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cd0 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.267645:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:1.0:1787826386.267647:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826386.267650:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece200, rc 0) 00000100:00100000:1.0:1787826386.267739:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:227600:x1874663332707456:12345-192.168.203.56@tcp:101:touch.0 Request processed in 9977us (14889us total) trans 8589957512 rc 0/0 00000100:00100000:1.0:1787826386.267747:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68734 00000100:00100000:2.0:1787826386.273268:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826386.274700:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332707712 00000100:00100000:1.0:1787826386.274716:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332707712 00000100:00100000:1.0:1787826386.275035:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68735 00000100:00100000:1.0:1787826386.275048:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:227600:x1874663332707712:12345-192.168.203.56@tcp:36:touch.0 00010000:00010000:1.0:1787826386.275150:0:11629:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826386.275185:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4c9db lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cd0 expref: 9 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.275211:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7845cb1900) 00010000:00010000:1.0:1787826386.275225:0:11629:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826386.275286:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.275299:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.275309:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826386.276680:0:11629:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000048395b4d reply state 000000005e4fe767 transno 8589957513 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.276705:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826386.276713:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:1.0:1787826386.276717:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826386.276720:0:11629:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787826386.277637:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:227600:x1874663332707712:12345-192.168.203.56@tcp:36:touch.0 Request processed in 2584us (4377us total) trans 8589957513 rc 0/0 00000100:00100000:1.0:1787826386.277647:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68735 00000001:00080000:0.0:1787826386.281246:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957512 is committed 00000001:00080000:0.0:1787826386.281271:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957511 is committed 00010000:00010000:3.0:1787826386.283409:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.283443:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.283452:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.283459:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826386.283465:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.283475:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.283500:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x508acae2 (ffff9f7845cb1a00) 00010000:00010000:3.0:1787826386.283503:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826386.283511:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.283518:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4c9d4 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826386.284989:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826386.285052:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332707840 00000100:00100000:3.0:1787826386.285066:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332707840 00000100:00100000:3.0:1787826386.285170:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68736 00000100:00100000:3.0:1787826386.285184:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:227600:x1874663332707840:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:3.0:1787826386.285219:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826386.285243:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c9f0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cd7 expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.285319:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c9f0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6e8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cd7 expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.285336:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c9f0 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6e8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cd7 expref: 9 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826386.285361:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:3.0:1787826386.285364:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1787826386.285368:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826386.285573:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:227600:x1874663332707840:12345-192.168.203.56@tcp:101:touch.0 Request processed in 391us (595us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.285581:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68736 00000100:00100000:3.0:1787826386.288384:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826386.289620:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332707968 00000100:00100000:0.0:1787826386.289637:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332707968 00000100:00100000:0.0:1787826386.289706:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30902 00000100:00100000:0.0:1787826386.289717:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:0+-99:2392:x1874663332707968:12345-192.168.203.56@tcp:8:touch.0 00010000:00080000:0.0:1787826386.289759:0:20167:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from 398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp t4294972907 exp 0000000000000000 cur 8198 last 0 00000020:00000080:0.0:1787826386.289799:0:20167:0:(genops.c:1385:class_connect()) connect: client 398e0287-d3c7-44dd-b1d9-6b85f698e615, cookie 0xe21309aa81f4c9f7 00002000:00100000:0.0:1787826386.289813:0:20167:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli 398e0287-d3c7-44dd-b1d9-6b85f698e615/ffff9f78435be800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113950 ocd_grant: 8351744 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1787826386.289834:0:20167:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:0.0:1787826386.289912:0:20167:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:0.0:1787826386.289951:0:20167:0:(import.c:65:import_set_state_nolock()) ffff9f78435b8000 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1787826386.290049:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2392:x1874663332707968:12345-192.168.203.56@tcp:8:touch.0 Request processed in 332us (1680us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826386.290058:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30902 00000100:00100000:3.0:1787826386.301429:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826386.301525:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332708096 00000100:00100000:2.0:1787826386.301538:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332708096 00000100:00100000:2.0:1787826386.301641:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30903 00000100:00100000:2.0:1787826386.301653:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332708096:12345-192.168.203.56@tcp:2:touch.0 00000100:00080000:3.0:1787826386.302082:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1787826386.304291:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973867 is committed 00000001:00100000:2.0:1787826386.304308:0:8438:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000001:00100000:2.0:1787826386.304315:0:8438:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:2.0:1787826386.304319:0:8438:0:(genops.c:885:class_export_put()) final put ffff9f784357b000/398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:3.0:1787826386.304393:0:128839:0:(genops.c:844:class_export_destroy()) destroying export ffff9f784357b000/398e0287-d3c7-44dd-b1d9-6b85f698e615 for lustre-OST0000 00000020:00000080:3.0:1787826386.304467:0:128839:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9f78435ba000 for lustre-OST0000 00000100:00100000:0.0:1787826386.304630:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:0:x1874663332708096:12345-192.168.203.56@tcp:2:touch.0 Request processed in 2972us (3205us total) trans 4294973867 rc 0/0 00000100:00100000:0.0:1787826386.304642:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30903 00000100:00100000:3.0:1787826386.307379:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826386.307484:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332708352 00000100:00100000:0.0:1787826386.307497:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332708352 00000100:00100000:0.0:1787826386.307568:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9408 00000100:00100000:0.0:1787826386.307580:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:227600:x1874663332708352:12345-192.168.203.56@tcp:35:touch.0 00000100:00100000:0.0:1787826386.307931:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848cfca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:227600:x1874663332708352:12345-192.168.203.56@tcp:35:touch.0 Request processed in 351us (563us total) trans 8589957514 rc 0/0 00000100:00100000:0.0:1787826386.307941:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9408 00000100:00100000:3.0:1787826386.925861:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346370944:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.925949:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926009:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346371456:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926039:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926059:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346371968:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926085:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926131:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346372480:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926158:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926185:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1787826386.926192:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346371328:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926214:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926232:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346371840:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926253:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926273:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c321f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346372352:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926295:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926327:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346371072:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926353:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926372:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346371584:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926393:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926410:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346372096:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926431:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926451:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1787826386.926457:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346371200:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926479:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926496:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346371712:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926515:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926529:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346372224:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926545:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826386.926578:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346370944 00000100:00100000:3.0:1787826386.926586:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346370944 00000100:00100000:3.0:1787826386.926645:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2251 00000100:00100000:3.0:1787826386.926655:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346370944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926715:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346370944:12345-0@lo:400:kworker.0 Request processed in 60us (766us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.926722:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2251 00000100:00100000:3.0:1787826386.926759:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346371456 00000100:00100000:3.0:1787826386.926763:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346371456 00000100:00100000:3.0:1787826386.926799:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346372480 00000100:00100000:3.0:1787826386.926802:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346372480 00000100:00100000:3.0:1787826386.926820:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346371328 00000100:00100000:3.0:1787826386.926823:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346371328 00000100:00100000:3.0:1787826386.926843:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346371840 00000100:00100000:3.0:1787826386.926845:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346371840 00000100:00100000:3.0:1787826386.926865:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346371200 00000100:00100000:3.0:1787826386.926868:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346371200 00000100:00100000:3.0:1787826386.926885:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346371712 00000100:00100000:3.0:1787826386.926888:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346371712 00000100:00100000:3.0:1787826386.926910:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14423 00000100:00100000:3.0:1787826386.926916:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346371456:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926948:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346371456:12345-0@lo:400:kworker.0 Request processed in 33us (909us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.926955:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14423 00000100:00100000:3.0:1787826386.926966:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14424 00000100:00100000:3.0:1787826386.926971:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346372480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.926996:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346372480:12345-0@lo:400:kworker.0 Request processed in 26us (838us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927001:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14424 00000100:00100000:3.0:1787826386.927011:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14425 00000100:00100000:3.0:1787826386.927016:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346371328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927038:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346371328:12345-0@lo:400:kworker.0 Request processed in 22us (824us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927043:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14425 00000100:00100000:3.0:1787826386.927052:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14426 00000100:00100000:3.0:1787826386.927057:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346371840:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927079:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346371840:12345-0@lo:400:kworker.0 Request processed in 22us (825us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927084:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14426 00000100:00100000:3.0:1787826386.927093:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14427 00000100:00100000:3.0:1787826386.927098:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346371200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927163:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346371200:12345-0@lo:400:kworker.0 Request processed in 64us (682us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927169:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14427 00000100:00100000:3.0:1787826386.927179:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14428 00000100:00100000:3.0:1787826386.927185:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346371712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927208:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346371712:12345-0@lo:400:kworker.0 Request processed in 24us (692us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927213:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14428 00000100:00100000:3.0:1787826386.927234:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346371968 00000100:00100000:3.0:1787826386.927238:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346371968 00000100:00100000:3.0:1787826386.927265:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346372352 00000100:00100000:3.0:1787826386.927267:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346372352 00000100:00100000:3.0:1787826386.927284:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346371584 00000100:00100000:3.0:1787826386.927287:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346371584 00000100:00100000:3.0:1787826386.927302:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346372224 00000100:00100000:3.0:1787826386.927304:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346372224 00000100:00100000:3.0:1787826386.927323:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8916 00000100:00100000:3.0:1787826386.927329:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346371968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927360:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7854627800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346371968:12345-0@lo:400:kworker.0 Request processed in 32us (1275us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927366:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8916 00000100:00100000:3.0:1787826386.927377:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8917 00000100:00100000:3.0:1787826386.927383:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346372352:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927409:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346372352:12345-0@lo:400:kworker.0 Request processed in 27us (1113us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927415:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8917 00000100:00100000:3.0:1787826386.927425:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8918 00000100:00100000:3.0:1787826386.927430:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346371584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927457:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346371584:12345-0@lo:400:kworker.0 Request processed in 26us (1063us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927463:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8918 00000100:00100000:3.0:1787826386.927473:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8919 00000100:00100000:3.0:1787826386.927478:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346372224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927506:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d009f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346372224:12345-0@lo:400:kworker.0 Request processed in 28us (959us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927511:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8919 00000100:00100000:3.0:1787826386.927546:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346371072 00000100:00100000:3.0:1787826386.927549:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346371072 00000100:00100000:3.0:1787826386.927574:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346372096 00000100:00100000:3.0:1787826386.927577:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346372096 00000100:00100000:3.0:1787826386.927593:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24245 00000100:00100000:3.0:1787826386.927599:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346371072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927631:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346371072:12345-0@lo:400:kworker.0 Request processed in 32us (1277us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927636:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24245 00000100:00100000:3.0:1787826386.927646:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24246 00000100:00100000:3.0:1787826386.927651:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346372096:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927677:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cdb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346372096:12345-0@lo:400:kworker.0 Request processed in 26us (1245us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826386.927682:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24246 00000100:00100000:3.0:1787826386.927748:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346371328:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927760:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346371840:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927771:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c321f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346372352:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927780:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b307b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346370944:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927789:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346371456:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927801:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346371968:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927813:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346372480:0@lo:400:kworker.0 00000100:00080000:3.0:1787826386.927852:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826386.927856:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826386.927860:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1787826386.927862:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826386.927901:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346371200:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927911:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784adf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346371712:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927921:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346372224:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927930:0:3654:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 8589957511 gen 2 00000004:00080000:3.0:1787826386.927935:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957509 committed transno 8589957511 00000004:00080000:3.0:1787826386.927954:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957510 committed transno 8589957511 00000004:00080000:3.0:1787826386.927967:0:3654:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 8589957511 committed transno 8589957511 00000100:00100000:3.0:1787826386.927980:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346371072:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.927993:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346371584:0@lo:400:kworker.0 00000100:00100000:3.0:1787826386.928002:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346372096:0@lo:400:kworker.0 00010000:00010000:0.0:1787826387.116417:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826387.437519:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346372608:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826387.437611:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826387.438213:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346372608 00000100:00100000:1.0:1787826387.438225:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346372608 00000100:00100000:1.0:1787826387.438297:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6283 00000100:00100000:1.0:1787826387.438309:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346372608:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826387.438348:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826387.438485:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784adf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346372608:12345-0@lo:13:osp-pre-1-1.0 Request processed in 177us (873us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826387.438493:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6283 00000100:00100000:3.0:1787826387.438586:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346372608:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826387.680456:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826387.680536:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332708480 00000100:00100000:3.0:1787826387.680549:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332708480 00000100:00100000:3.0:1787826387.680620:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68737 00000100:00100000:3.0:1787826387.680634:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:227663:x1874663332708480:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:3.0:1787826387.680670:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826387.680697:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c9fe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cde expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.680778:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca05 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.680796:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca05 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.680811:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca05 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826387.680871:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826387.680875:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826387.680900:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca05 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.680928:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4c9fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cde expref: 12 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.680948:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca05 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cde expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.680958:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826387.680961:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826387.680966:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826387.681039:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:227663:x1874663332708480:12345-192.168.203.56@tcp:101:link.0 Request processed in 407us (598us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826387.681047:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68737 00000100:00100000:3.0:1787826387.688637:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826387.689158:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332708608 00000100:00100000:1.0:1787826387.689174:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332708608 00000100:00100000:1.0:1787826387.689249:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68738 00000100:00100000:1.0:1787826387.689263:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:227663:x1874663332708608:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:1.0:1787826387.689295:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826387.689323:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4ca0c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801ce5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689408:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689427:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689440:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689458:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689465:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689471:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689589:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ca21 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689599:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ca21 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689610:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ca21 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826387.689656:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6e8:0x0]: returning size 0 00000004:00200000:1.0:1787826387.689661:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826387.689682:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ca21 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689695:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ca21 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689704:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689711:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689718:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689724:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689731:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecf600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826387.689735:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689742:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689757:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826387.689761:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826387.689765:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689772:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca13 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689787:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689794:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689800:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689805:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826387.689808:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689813:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689820:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x508acae2 (ffff9f7961614200) 00010000:00010000:1.0:1787826387.689822:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826387.689825:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689830:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ca1a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689866:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4ca0c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ce5 expref: 13 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689888:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ca21 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ce5 expref: 12 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826387.689896:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:1.0:1787826387.689899:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826387.689901:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826387.689988:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:227663:x1874663332708608:12345-192.168.203.56@tcp:101:link.0 Request processed in 728us (1365us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826387.689997:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68738 00000100:00100000:1.0:1787826387.696706:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826387.696860:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332708864 00000100:00100000:3.0:1787826387.696874:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332708864 00000100:00100000:3.0:1787826387.696949:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68739 00000100:00100000:3.0:1787826387.697078:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:227663:x1874663332708864:12345-192.168.203.56@tcp:36:link.0 00010000:00010000:3.0:1787826387.697278:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1787826387.697312:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ca21 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801ce5 expref: 12 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.697332:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:3.0:1787826387.697344:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca05 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cde expref: 11 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.697354:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826387.697357:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826387.697387:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.697403:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.697415:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.697433:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.697441:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826387.697447:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00020000:3.0:1787826387.697456:0:6528:0:(mdt_handler.c:4171:mdt_object_pdo_lock()) cfs_fail_timeout id 145 sleeping for 15000ms 00000100:00100000:3.0:1787826387.953076:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79465ca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346372736:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826387.953331:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346372992:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826387.953412:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826387.953492:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826387.953501:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977d26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346372864:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826387.953534:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826387.953635:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346372992 00000100:00100000:1.0:1787826387.953651:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346372992 00000100:00100000:1.0:1787826387.953728:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24247 00000100:00100000:1.0:1787826387.953739:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346372992:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826387.953779:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:1.0:1787826387.953865:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346372992:12345-0@lo:41:osp-pre-0-1.0 Request processed in 128us (452us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826387.953872:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24247 00000100:00100000:1.0:1787826387.953901:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346372864 00000100:00100000:1.0:1787826387.953905:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346372864 00000100:00100000:1.0:1787826387.953928:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6284 00000100:00100000:1.0:1787826387.953934:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346372864:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826387.953952:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826387.953981:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346372864:12345-0@lo:13:osp-pre-0-1.0 Request processed in 48us (447us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826387.953990:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6284 00000100:00100000:1.0:1787826387.954040:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977d26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346372864:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826387.954055:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c5b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346372992:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826387.954346:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826387.954381:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b75bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346373248:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826387.954409:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826387.956838:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346373120:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826387.956954:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826387.957092:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346372736 00000100:00100000:2.0:1787826387.957152:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346372736 00000100:00100000:2.0:1787826387.957218:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6285 00000100:00100000:2.0:1787826387.957230:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346372736:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826387.957261:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826387.957311:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346372736:12345-0@lo:13:osp-pre-1-0.0 Request processed in 83us (2968us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826387.957319:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6285 00000100:00100000:3.0:1787826387.957918:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79465ca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346372736:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826387.959996:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346373248 00000100:00100000:2.0:1787826387.960015:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346373248 00000100:00100000:2.0:1787826387.960090:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6286 00000100:00100000:2.0:1787826387.960132:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346373248:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826387.960169:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826387.960275:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346373248:12345-0@lo:13:osp-pre-0-0.0 Request processed in 173us (5864us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826387.960283:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6286 00000100:00100000:0.0:1787826387.961039:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346373120 00000100:00100000:0.0:1787826387.961056:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346373120 00000100:00100000:0.0:1787826387.961158:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24248 00000100:00100000:0.0:1787826387.961170:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346373120:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1787826387.961209:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:0.0:1787826387.961516:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346373120:12345-0@lo:41:osp-pre-1-0.0 Request processed in 304us (4519us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826387.961527:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24248 00000100:00100000:0.0:1787826387.961615:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346373120:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826387.962144:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b75bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346373248:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1787826388.140437:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826388.194540:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826388.194675:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332708992 00000100:00100000:0.0:1787826388.194685:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332708992 00000100:00100000:0.0:1787826388.194769:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68740 00000100:00100000:0.0:1787826388.194782:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:227665:x1874663332708992:12345-192.168.203.56@tcp:101:mkdir.0 00010000:00010000:0.0:1787826388.194803:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826388.194827:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4ca36 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cec expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826388.194898:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826388.194911:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 2/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826388.194919:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826388.194938:0:6529:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826388.194949:0:6529:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826388.195057:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787826389.164360:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00020000:3.0:1787826389.829198:0:6528:0:(mdt_handler.c:4171:mdt_object_pdo_lock()) cfs_fail_timeout interrupted 00010000:00010000:3.0:1787826389.856964:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.856986:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.857001:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826389.857331:0:6528:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000080a993b7 reply state 00000000ed94b013 transno 8589957515 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.857345:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.857352:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:3.0:1787826389.857356:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00000100:00100000:3.0:1787826389.857360:0:6528:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826389.857367:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.860347:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.860357:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.860365:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862368:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826389.862387:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862397:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862416:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826389.862419:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826389.862424:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862434:0:6528:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862442:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862490:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862499:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca28 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826389.862521:0:6528:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000080a993b7 reply state 00000000ed94b013 transno 8589957515 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862532:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826389.862537:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x496ac7f5 (ffff9f79460a2300) 00010000:00010000:3.0:1787826389.862539:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826389.862543:0:6528:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:3.0:1787826389.862658:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:227663:x1874663332708864:12345-192.168.203.56@tcp:36:link.0 Request processed in 2165581us (2165960us total) trans 8589957515 rc 0/0 00000100:00100000:3.0:1787826389.862667:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68739 00010000:00010000:0.0:1787826389.863180:0:6529:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826389.863190:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826389.863256:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826389.863260:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826389.863284:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826389.863318:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4ca36 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826389.863336:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826389.863344:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826389.863346:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826389.863349:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826389.863399:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:227665:x1874663332708992:12345-192.168.203.56@tcp:101:mkdir.0 Request processed in 1668620us (1668869us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826389.863406:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68740 00000100:00100000:1.0:1787826389.868435:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826389.868470:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826389.868537:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332709120 00000100:00100000:0.0:1787826389.868548:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332709120 00000100:00100000:0.0:1787826389.868622:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332709376 00000100:00100000:0.0:1787826389.868625:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332709376 00000100:00100000:0.0:1787826389.868651:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68741 00000100:00100000:0.0:1787826389.868663:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332709120:12345-192.168.203.56@tcp:41:mkdir.0 00000004:00000024:0.0:1787826389.868702:0:35098:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:0.0:1787826389.868773:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c5b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332709120:12345-192.168.203.56@tcp:41:mkdir.0 Request processed in 112us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826389.868780:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68741 00000100:00100000:0.0:1787826389.868802:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68742 00000100:00100000:0.0:1787826389.868808:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:227665:x1874663332709376:12345-192.168.203.56@tcp:36:mkdir.0 00000100:00100000:0.0:1787826389.868906:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346373376:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826389.868955:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826389.869009:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7943297b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826389.869037:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346373376 00000100:00100000:0.0:1787826389.869041:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346373376 00000100:00100000:0.0:1787826389.869069:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24249 00000100:00100000:0.0:1787826389.869076:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:35098:x1874663346373376:12345-0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826389.869166:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:35098:x1874663346373376:12345-0@lo:1000:mdt00_013.0 Request processed in 91us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826389.869174:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24249 00000100:00100000:0.0:1787826389.869302:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78508c2680 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346373376:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826389.869352:0:35098:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346373504:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826389.869377:0:35098:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826389.869396:0:35098:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78453d9900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826389.869410:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346373504 00000100:00100000:0.0:1787826389.869413:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346373504 00000100:00100000:0.0:1787826389.869439:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24250 00000100:00100000:0.0:1787826389.869446:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:35098:x1874663346373504:12345-0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826389.869562:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e763800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:35098:x1874663346373504:12345-0@lo:1000:mdt00_013.0 Request processed in 116us (183us total) trans 0 rc 1/1 00000100:00100000:0.0:1787826389.869569:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24250 00000100:00100000:0.0:1787826389.869599:0:35098:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e762680 pname:cluuid:pid:xid:nid:opc:job mdt00_013:lustre-MDT0001-mdtlov_UUID:35098:1874663346373504:0@lo:1000:mdt00_013.0 00000100:00100000:0.0:1787826389.869651:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:227665:x1874663332709376:12345-192.168.203.56@tcp:36:mkdir.0 Request processed in 842us (1178us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787826389.869658:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68742 00000100:00080000:1.0:1787826389.869788:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1787826390.189779:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826390.525302:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826390.525467:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332709504 00000100:00100000:3.0:1787826390.525477:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332709504 00000100:00100000:0.0:1787826390.525646:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68743 00000100:00100000:0.0:1787826390.525661:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d817480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:227347:x1874663332709504:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826390.525687:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826390.525711:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ca4b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cf3 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.525794:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ca52 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.525808:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ca52 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.525820:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ca52 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826390.525878:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826390.525882:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826390.525903:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ca52 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.525928:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4ca4b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cf3 expref: 14 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.525946:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ca52 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cf3 expref: 13 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.525954:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826390.525957:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826390.525960:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826390.526025:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d817480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:227347:x1874663332709504:12345-192.168.203.56@tcp:101:bash.0 Request processed in 369us (734us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826390.526032:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68743 00000100:00080000:1.0:1787826390.527449:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826390.530940:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826390.531441:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332709632 00000100:00100000:2.0:1787826390.531459:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332709632 00000100:00100000:2.0:1787826390.531523:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9409 00000100:00100000:2.0:1787826390.531534:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:227347:x1874663332709632:12345-192.168.203.56@tcp:37:bash.0 00000100:00100000:0.0:1787826390.538485:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:227347:x1874663332709632:12345-192.168.203.56@tcp:37:bash.0 Request processed in 6942us (7544us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826390.538499:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9409 00000100:00100000:1.0:1787826390.550768:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826390.554722:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332709760 00000100:00100000:3.0:1787826390.554738:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332709760 00000100:00100000:3.0:1787826390.554824:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68744 00000100:00100000:3.0:1787826390.554835:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d816a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:227707:x1874663332709760:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826390.554854:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826390.555122:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca59 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801cfa expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555192:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555205:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555217:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555227:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555234:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555240:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555332:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4ca6e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555341:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4ca6e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555348:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4ca6e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826390.555387:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6e8:0x0]: returning size 0 00000004:00200000:3.0:1787826390.555391:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826390.555409:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4ca6e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555420:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4ca6e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555430:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555580:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555588:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555596:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555603:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826390.555608:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555617:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555630:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826390.555633:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826390.555637:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555644:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4ca60 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555658:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555664:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555669:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555675:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826390.555678:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555683:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555690:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x496ac7f5 (ffff9f79460a2300) 00010000:00010000:3.0:1787826390.555692:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826390.555695:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555701:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4ca67 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555726:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca59 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cfa expref: 15 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555738:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4ca6e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cfa expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.555745:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:3.0:1787826390.555748:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826390.555750:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826390.555866:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d816a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:227707:x1874663332709760:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1032us (5108us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826390.556670:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68744 00000100:00100000:3.0:1787826390.560960:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826390.561147:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332709888 00000100:00100000:1.0:1787826390.561158:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332709888 00000100:00100000:1.0:1787826390.561240:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30904 00000100:00100000:1.0:1787826390.561253:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332709888:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826390.561271:0:35096:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826390.561302:0:35096:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d01 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787826390.561330:0:35096:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x524:0x0] initial LVB size: 0, mtime: 0x6a9010d1, atime: 0x6a9010d1, ctime: 0x6a9010d1, blocks: 0x0 00010000:00010000:1.0:1787826390.561350:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x524:0x0].0x0 since=8202s v=(0 0) 00010000:00010000:1.0:1787826390.561356:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x524:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826390.561374:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 6 pid: 35096 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.561394:0:35096:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 6 pid: 35096 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.561407:0:35096:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826390.561467:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79465ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332709888:12345-192.168.203.56@tcp:101:rm.0 Request processed in 217us (517us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826390.561473:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30904 00000100:00080000:3.0:1787826390.561612:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826390.572321:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826390.572655:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332710144 00000100:00100000:3.0:1787826390.572665:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332710144 00000100:00100000:3.0:1787826390.572730:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68745 00000100:00100000:3.0:1787826390.572742:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:227707:x1874663332710144:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826390.572801:0:35092:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787826390.572832:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4ca6e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cfa expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.572856:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4c9f0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6e8:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cd7 expref: 13 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.572868:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:3.0:1787826390.572877:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ca52 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cf3 expref: 12 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.572897:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826390.572901:0:35092:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826390.572925:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.572936:0:35092:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.572944:0:35092:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.574557:0:35092:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.574584:0:35092:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 6 pid: 6529 timeout: 8302 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.574606:0:35092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346373632:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826390.574729:0:35092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7846318b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826390.583631:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826390.583743:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332710272 00000100:00100000:3.0:1787826390.583752:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332710272 00010000:00010000:3.0:1787826390.583811:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 7 pid: 6529 timeout: 8302 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826390.583853:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22092 00000100:00100000:3.0:1787826390.583863:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332710272:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826390.583884:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826390.583891:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 7 pid: 6529 timeout: 8302 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.583904:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 7 pid: 6529 timeout: 8302 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.583926:0:6511:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826390.583931:0:6511:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.583942:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.583949:0:6511:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.583967:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826390.584021:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332710272:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 160us (409us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826390.584028:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22092 00010000:00010000:2.0:1787826390.584483:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4ca3d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801cec expref: 6 pid: 6529 timeout: 8302 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.584499:0:35092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job mdt00_009:lustre-MDT0000_UUID:35092:1874663346373632:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826390.584529:0:35092:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584537:0:35092:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584543:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584561:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584570:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584578:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584686:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584694:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584701:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584954:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584962:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.584968:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826390.586074:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1787826390.586691:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826390.586698:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586707:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586722:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:2.0:1787826390.586725:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787826390.586730:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586736:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca8a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586768:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586774:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586781:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586788:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586793:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826390.586796:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586802:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586809:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826390.586812:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826390.586814:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586820:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4ca7c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826390.586834:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000006fbf879 reply state 00000000845ef0fa transno 8589957516 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586846:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.586853:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x496ac7f5 (ffff9f79460a2300) 00010000:00010000:2.0:1787826390.586856:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826390.586859:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787826390.587521:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:227707:x1874663332710144:12345-192.168.203.56@tcp:36:rm.0 Request processed in 14767us (15197us total) trans 8589957516 rc 0/0 00000100:00100000:2.0:1787826390.587532:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68745 00000100:00100000:1.0:1787826390.606799:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826390.611138:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332710400 00000100:00100000:2.0:1787826390.611157:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332710400 00000100:00100000:2.0:1787826390.611302:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68746 00000100:00100000:2.0:1787826390.611314:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:227707:x1874663332710400:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826390.611342:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826390.611374:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca91 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d08 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.611448:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4ca98 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.611465:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4ca98 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.611477:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4ca98 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826390.611537:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826390.611541:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826390.611564:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4ca98 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.611591:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ca91 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d08 expref: 15 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.611613:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4ca98 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d08 expref: 14 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.611639:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826390.611642:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826390.611647:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eec00, rc 0) 00000100:00100000:2.0:1787826390.611732:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:227707:x1874663332710400:12345-192.168.203.56@tcp:101:rm.0 Request processed in 419us (4942us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826390.611740:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68746 00000100:00100000:3.0:1787826390.618148:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826390.623482:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332710528 00000100:00100000:1.0:1787826390.623497:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332710528 00000100:00100000:1.0:1787826390.626954:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68747 00000100:00100000:1.0:1787826390.626969:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:227707:x1874663332710528:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826390.626990:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826390.627012:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca9f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d0f expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627082:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627120:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627134:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627151:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627164:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627171:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627257:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cab4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627267:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cab4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627275:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cab4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826390.627315:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6e8:0x0]: returning size 0 00000004:00200000:1.0:1787826390.627320:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826390.627338:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cab4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627349:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cab4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627359:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627943:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627952:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627963:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627969:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826390.627974:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627983:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.627999:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826390.628002:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826390.628008:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628016:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4caa6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628035:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628042:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628047:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628052:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826390.628055:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628065:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628073:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x508acae2 (ffff9f7961614000) 00010000:00010000:1.0:1787826390.628075:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826390.628078:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628083:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4caad lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628208:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf600/0xe21309aa81f4ca9f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d0f expref: 16 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628225:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cab4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d0f expref: 15 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.628233:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:1.0:1787826390.628236:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826390.628238:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826390.628359:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:227707:x1874663332710528:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1392us (10264us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826390.628377:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68747 00000100:00100000:3.0:1787826390.637264:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826390.638767:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332710784 00000100:00100000:0.0:1787826390.638782:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332710784 00000100:00100000:0.0:1787826390.638867:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68748 00000100:00100000:0.0:1787826390.638887:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:227707:x1874663332710784:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826390.638947:0:35095:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1787826390.638981:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cab4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d0f expref: 15 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639007:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:0.0:1787826390.639019:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4ca98 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d08 expref: 14 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639030:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826390.639033:0:35095:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826390.639061:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639074:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639083:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639129:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x508acae2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639141:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639147:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639246:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639255:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.639263:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787826390.640338:0:35095:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].218, 64 off22144 00000004:00080000:0.0:1787826390.640350:0:35095:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8202852243711ns, next at 8203852240791ns, rc = 0 00010000:00010000:0.0:1787826390.640607:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.640740:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010080000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.640752:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40010080000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.640759:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40010080000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.640766:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010480000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.640776:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010480000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641663:0:35095:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826390.641693:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641701:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641708:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641712:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826390.641717:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641724:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641736:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:0.0:1787826390.641739:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826390.641743:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641748:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cac9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6e8:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641771:0:35095:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641782:0:35095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641787:0:35095:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641793:0:35095:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641798:0:35095:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826390.641800:0:35095:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641806:0:35095:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641815:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826390.641817:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826390.641820:0:35095:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641826:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cabb lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826390.641835:0:35095:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f44abb0c reply state 00000000b674c002 transno 8589957517 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641845:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.641851:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x508acae2 (ffff9f794b57a800) 00010000:00010000:0.0:1787826390.641853:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826390.641857:0:35095:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787826390.642157:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:227707:x1874663332710784:12345-192.168.203.56@tcp:36:rm.0 Request processed in 3277us (4898us total) trans 8589957517 rc 0/0 00000100:00100000:0.0:1787826390.642165:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68748 00010000:00010000:2.0:1787826390.650156:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826390.650170:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.650183:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826390.650188:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.650199:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.650218:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:2.0:1787826390.650221:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826390.650226:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.650233:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cad0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6e8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.650248:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826390.650250:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826390.650252:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826391.212306:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826391.260239:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826391.260284:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826391.260353:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826391.260482:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826391.260600:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332710912 00000100:00100000:3.0:1787826391.260610:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332710912 00000100:00100000:3.0:1787826391.260666:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2252 00000100:00100000:3.0:1787826391.260677:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332710912:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826391.260748:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332710912:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 72us (523us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826391.260757:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2252 00000100:00100000:1.0:1787826391.260958:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332711424 00000100:00100000:1.0:1787826391.260963:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332711424 00000100:00100000:1.0:1787826391.261011:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8920 00000100:00100000:1.0:1787826391.261021:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332711424:12345-192.168.203.56@tcp:400:kworker.0 00000100:00080000:3.0:1787826391.265562:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826391.266658:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332711424:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 5626us (6291us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826391.266672:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8920 00000100:00100000:2.0:1787826391.268254:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826391.268296:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826391.268352:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332711936 00000100:00100000:1.0:1787826391.268362:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332711936 00000100:00100000:1.0:1787826391.268434:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8921 00000100:00100000:1.0:1787826391.268444:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332711936:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826391.268517:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332711936:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 74us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826391.268525:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8921 00000100:00080000:1.0:1787826391.268549:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826391.276549:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332711168 00000100:00100000:0.0:1787826391.276565:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332711168 00000100:00100000:0.0:1787826391.276647:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332711680 00000100:00100000:0.0:1787826391.276651:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332711680 00000100:00100000:0.0:1787826391.276684:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14429 00000100:00100000:0.0:1787826391.276695:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332711168:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826391.276778:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332711168:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 85us (16493us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826391.276786:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14429 00000100:00100000:0.0:1787826391.276806:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14430 00000100:00100000:0.0:1787826391.276813:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332711680:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826391.276836:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332711680:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (16355us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826391.276843:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14430 00000100:00080000:1.0:1787826391.277400:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826391.278281:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826391.278313:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826391.278337:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826391.278494:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332711296 00000100:00100000:1.0:1787826391.278502:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332711296 00000100:00100000:1.0:1787826391.278544:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332711808 00000100:00100000:1.0:1787826391.278547:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332711808 00000100:00100000:1.0:1787826391.278574:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8922 00000100:00100000:1.0:1787826391.278584:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332711296:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826391.278644:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332711296:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 61us (370us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826391.278651:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8922 00000100:00100000:1.0:1787826391.278666:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8923 00000100:00100000:1.0:1787826391.278672:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332711808:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826391.278694:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332711808:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 22us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826391.278704:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8923 00000100:00080000:1.0:1787826391.278725:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826391.278736:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332711040 00000100:00100000:1.0:1787826391.278739:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332711040 00000100:00100000:1.0:1787826391.278763:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332711552 00000100:00100000:1.0:1787826391.278766:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332711552 00000100:00100000:1.0:1787826391.278783:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14431 00000100:00100000:1.0:1787826391.278788:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:2394:x1874663332711040:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826391.278818:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:2394:x1874663332711040:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (10521us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826391.278825:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14431 00000100:00100000:1.0:1787826391.278836:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14432 00000100:00100000:1.0:1787826391.278842:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332711552:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826391.278863:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332711552:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 21us (550us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826391.278870:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14432 00000001:00080000:0.0:1787826391.342434:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957517 is committed 00000004:00080000:0.0:1787826391.342450:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 217 committed 00010000:00010000:2.0:1787826391.342482:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342495:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342503:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342509:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826391.342514:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342522:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342539:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:2.0:1787826391.342541:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826391.342546:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342552:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4c9e9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342577:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342584:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342590:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342595:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826391.342597:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342603:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342609:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x496ac7f5 (ffff9f79460a2300) 00010000:00010000:2.0:1787826391.342612:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826391.342614:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342620:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4ca2f lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342628:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342633:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342639:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342645:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826391.342647:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342653:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342659:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e8:0x0].0x0 (ffff9f7961614a00) 00010000:00010000:2.0:1787826391.342661:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00010000:00010000:2.0:1787826391.342663:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342669:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4ca44 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x6e8:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342685:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342691:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342697:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342702:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826391.342705:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342711:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342722:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x496ac7f5 (ffff9f79460a2300) 00010000:00010000:2.0:1787826391.342724:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826391.342727:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342732:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4ca83 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x496ac7f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342746:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342751:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342757:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342762:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826391.342764:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342771:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342779:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x508acae2 (ffff9f794b57a800) 00010000:00010000:2.0:1787826391.342781:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826391.342784:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.342789:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cac2 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x508acae2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826391.343193:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346373760:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1787826391.343274:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826391.343487:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346373760 00000100:00100000:1.0:1787826391.343495:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346373760 00000100:00100000:1.0:1787826391.343560:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30905 00000100:00100000:1.0:1787826391.343568:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78508c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346373760:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1787826391.343594:0:35096:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1316 count 1 00010000:00010000:1.0:1787826391.343616:0:35096:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826391.343635:0:35096:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 5 pid: 35096 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826391.343644:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x524:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1787826391.343648:0:35096:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x524:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826391.343653:0:35096:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826391.343677:0:35096:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 5 pid: 35096 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826391.343690:0:35096:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 5 pid: 35096 timeout: 8303 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826391.343700:0:35096:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:35096:1874663346373888:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826391.343769:0:35096:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784dedd300 going to sleep for 11 seconds 00000100:00080000:2.0:1787826391.343812:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826391.346045:0:35096:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f785189c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0000_UUID:35096:1874663346373888:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826391.346230:0:35096:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826391.346844:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826391.347023:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332712064 00000100:00100000:0.0:1787826391.347031:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332712064 00010000:00010000:0.0:1787826391.347084:0:6512:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 6 pid: 35096 timeout: 8303 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826391.347246:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22093 00000100:00100000:0.0:1787826391.347257:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332712064:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826391.347273:0:6512:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826391.347279:0:6512:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 6 pid: 35096 timeout: 8303 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.347293:0:6512:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 6 pid: 35096 timeout: 8303 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.347316:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071996532/0xe21309aa81f4ca75 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x524:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d01 expref: 6 pid: 35096 timeout: 8303 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.347333:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x524:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826391.347337:0:6512:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x524:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826391.347342:0:6512:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.347352:0:6512:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.348225:0:6512:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826391.349754:0:35096:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826391.349774:0:35096:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826391.349791:0:35096:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826391.349800:0:35096:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826391.349884:0:35096:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826391.349948:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332712064:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 2684us (3096us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826391.349961:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22093 00000100:00100000:1.0:1787826391.350157:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78508c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346373760:12345-0@lo:6:osp-syn-0-0.0 Request processed in 6587us (6880us total) trans 4294973868 rc 0/0 00000100:00100000:1.0:1787826391.350165:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30905 00000004:00080000:0.0:1787826391.350253:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7842c93100/2, rc 0, transno 4294973868 00000100:00100000:0.0:1787826391.350261:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346373760:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1787826391.350299:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826391.350307:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.350316:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826391.350320:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.350329:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.350346:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.350478:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae0f1415/0xe21309aa81f4cad7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x524:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 35096 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.350765:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826391.350767:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826391.350770:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826392.050517:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346374016:0@lo:400:kworker.0 00000100:00100000:2.0:1787826392.050595:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346374144:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.050625:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826392.050640:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826392.050702:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346374528:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.050736:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826392.051000:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346375040:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.051030:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826392.051052:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346375552:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.051127:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826392.051149:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1787826392.051156:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346374400:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.051179:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826392.051198:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346374912:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.051218:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826392.051234:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346375424:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.051256:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826392.051379:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346374016 00000100:00100000:1.0:1787826392.051387:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346374016 00000100:00100000:0.0:1787826392.052040:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346374528 00000100:00100000:0.0:1787826392.052047:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346374528 00000100:00100000:0.0:1787826392.052130:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346375552 00000100:00100000:0.0:1787826392.052134:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346375552 00000100:00100000:0.0:1787826392.052154:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346374400 00000100:00100000:0.0:1787826392.052157:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346374400 00000100:00100000:0.0:1787826392.052175:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346374912 00000100:00100000:0.0:1787826392.052178:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346374912 00000100:00100000:0.0:1787826392.052202:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14433 00000100:00100000:0.0:1787826392.052212:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346374528:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.052382:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2253 00000100:00100000:1.0:1787826392.052531:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346374016:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.052848:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346374016:12345-0@lo:400:kworker.0 Request processed in 458us (2224us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.052855:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2253 00000100:00100000:1.0:1787826392.052899:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14434 00000100:00100000:1.0:1787826392.052908:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346375552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.052956:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346375552:12345-0@lo:400:kworker.0 Request processed in 50us (1852us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.052962:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14434 00000100:00100000:1.0:1787826392.052980:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14435 00000100:00100000:1.0:1787826392.052986:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346374400:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.053015:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346374400:12345-0@lo:400:kworker.0 Request processed in 29us (1835us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.053021:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14435 00000100:00100000:1.0:1787826392.053032:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14436 00000100:00100000:1.0:1787826392.053038:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346374912:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.053067:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346374912:12345-0@lo:400:kworker.0 Request processed in 30us (1848us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.053083:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14436 00000100:00100000:1.0:1787826392.053129:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346375040 00000100:00100000:1.0:1787826392.053136:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346375040 00000100:00100000:1.0:1787826392.053165:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8924 00000100:00100000:1.0:1787826392.053172:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346375040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826392.053179:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346374656:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.053211:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346375040:12345-0@lo:400:kworker.0 Request processed in 39us (2179us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.053219:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8924 00000100:00100000:1.0:1787826392.053233:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346375424 00000100:00100000:2.0:1787826392.053233:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826392.053238:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346375424 00000100:00100000:1.0:1787826392.053274:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346374656 00000100:00100000:1.0:1787826392.053278:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346374656 00000100:00100000:1.0:1787826392.053301:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8925 00000100:00100000:1.0:1787826392.053308:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346375424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826392.054002:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346374144 00000100:00100000:3.0:1787826392.054010:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346374144 00000100:00100000:3.0:1787826392.054051:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24251 00000100:00100000:3.0:1787826392.054059:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346374144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.054729:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c15ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346375424:12345-0@lo:400:kworker.0 Request processed in 1418us (3468us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.054739:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8925 00000100:00100000:1.0:1787826392.054864:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8926 00000100:00100000:1.0:1787826392.054875:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346374656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.054914:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346374656:12345-0@lo:400:kworker.0 Request processed in 44us (1684us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.054986:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8926 00000100:00100000:1.0:1787826392.055063:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346374400:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.055174:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346374912:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.055186:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843641f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346375424:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.055198:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346375040:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.055211:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346375552:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.055224:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784c6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346374016:0@lo:400:kworker.0 00000100:00100000:0.0:1787826392.055598:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785189d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346374528:12345-0@lo:400:kworker.0 Request processed in 3386us (4859us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826392.055813:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14433 00000100:00080000:0.0:1787826392.055847:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787826392.055852:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787826392.055856:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787826392.055859:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787826392.056010:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346374528:0@lo:400:kworker.0 00000100:00100000:3.0:1787826392.056554:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346374144:12345-0@lo:400:kworker.0 Request processed in 2493us (5912us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826392.056563:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24251 00000100:00100000:3.0:1787826392.056597:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3655:1874663346374272:0@lo:400:kworker.0 00000100:00100000:3.0:1787826392.056883:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826392.057035:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346374784:0@lo:400:kworker.0 00000100:00100000:3.0:1787826392.057061:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826392.057930:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346375296:0@lo:400:kworker.0 00000100:00100000:3.0:1787826392.057974:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826392.058171:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346375296 00000100:00100000:3.0:1787826392.058179:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346375296 00000100:00100000:3.0:1787826392.058222:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8927 00000100:00100000:3.0:1787826392.058230:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346375296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826392.058293:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b307b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3655:x1874663346375296:12345-0@lo:400:kworker.0 Request processed in 64us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826392.058301:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8927 00000100:00100000:3.0:1787826392.058344:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843640380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346375296:0@lo:400:kworker.0 00000100:00100000:0.0:1787826392.059349:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346374272 00000100:00100000:0.0:1787826392.059368:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346374272 00000100:00100000:1.0:1787826392.062535:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346374784 00000100:00100000:1.0:1787826392.062550:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346374784 00000100:00100000:1.0:1787826392.062647:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14437 00000100:00100000:1.0:1787826392.062659:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0001_UUID+5:3655:x1874663346374272:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.063286:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0001_UUID+5:3655:x1874663346374272:12345-0@lo:400:kworker.0 Request processed in 628us (6401us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.063294:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14437 00000100:00100000:1.0:1787826392.063317:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14438 00000100:00100000:1.0:1787826392.063326:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346374784:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.063365:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346374784:12345-0@lo:400:kworker.0 Request processed in 32us (6294us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826392.063371:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14438 00000100:00100000:1.0:1787826392.063542:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3655:1874663346374272:0@lo:400:kworker.0 00000100:00100000:1.0:1787826392.063555:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346374784:0@lo:400:kworker.0 00000100:00100000:2.0:1787826392.063766:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346374656:0@lo:400:kworker.0 00000100:00100000:2.0:1787826392.063781:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346375168:0@lo:400:kworker.0 00000100:00100000:2.0:1787826392.063824:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826392.063859:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346375168 00000100:00100000:2.0:1787826392.063864:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346375168 00000100:00100000:2.0:1787826392.063903:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24252 00000100:00100000:2.0:1787826392.066128:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346375168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826392.066239:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346375168:12345-0@lo:400:kworker.0 Request processed in 2325us (2411us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826392.066250:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24252 00000100:00100000:2.0:1787826392.066315:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346375168:0@lo:400:kworker.0 00000100:00100000:2.0:1787826392.066355:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346374144:0@lo:400:kworker.0 00000100:00080000:0.0:1787826392.067058:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787826392.236346:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826392.556909:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826392.556943:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843434700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346375680:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826392.557027:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826392.557299:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346375680 00000100:00100000:0.0:1787826392.557309:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346375680 00000100:00100000:0.0:1787826392.557376:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6287 00000100:00100000:0.0:1787826392.557386:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346375680:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826392.557415:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826392.557473:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c323480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346375680:12345-0@lo:13:osp-pre-1-1.0 Request processed in 88us (445us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826392.557482:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6287 00000100:00100000:0.0:1787826392.557532:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843434700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346375680:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826393.068435:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826393.068463:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346375808:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826393.068561:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826393.068657:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346375936:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826393.068689:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826393.068712:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826393.068718:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346376064:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826393.068743:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826393.068777:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346375936 00000100:00100000:1.0:1787826393.068784:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346375936 00000100:00100000:1.0:1787826393.068836:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24253 00000100:00100000:1.0:1787826393.068846:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346375936:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826393.068878:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:1.0:1787826393.068933:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843640380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346375936:12345-0@lo:41:osp-pre-0-1.0 Request processed in 88us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826393.068939:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24253 00000100:00100000:1.0:1787826393.068985:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346375936:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826393.069338:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346375808 00000100:00100000:0.0:1787826393.069352:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346375808 00000100:00100000:0.0:1787826393.069390:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346376064 00000100:00100000:0.0:1787826393.069392:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346376064 00000100:00100000:0.0:1787826393.069405:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6288 00000100:00100000:0.0:1787826393.069411:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346375808:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787826393.069431:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826393.069471:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346375808:12345-0@lo:13:osp-pre-0-0.0 Request processed in 60us (911us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826393.069477:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6288 00000100:00100000:0.0:1787826393.069491:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6289 00000100:00100000:0.0:1787826393.069497:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346376064:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787826393.069505:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493197888 pending 0 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826393.069545:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843641f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346376064:12345-0@lo:13:osp-pre-0-1.0 Request processed in 48us (801us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826393.069551:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6289 00000100:00080000:3.0:1787826393.069660:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826393.069837:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787826393.069849:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784837b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346376192:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826393.069884:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826393.070003:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346376192 00000100:00100000:2.0:1787826393.070008:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346376192 00000100:00100000:2.0:1787826393.070045:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6290 00000100:00100000:2.0:1787826393.070052:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346376192:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826393.070064:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 493283904 pending 0 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826393.070197:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346376192:12345-0@lo:13:osp-pre-1-0.0 Request processed in 145us (311us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826393.070203:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6290 00000100:00100000:0.0:1787826393.070280:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784837b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346376192:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826393.070416:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346375808:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826393.070429:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787826393.070434:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d98ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346376320:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826393.070465:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826393.070526:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346376064:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826393.070825:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346376320 00000100:00100000:2.0:1787826393.070831:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346376320 00000100:00100000:2.0:1787826393.070877:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24254 00000100:00100000:2.0:1787826393.070884:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346376320:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826393.070909:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:2.0:1787826393.070972:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346376320:12345-0@lo:41:osp-pre-1-0.0 Request processed in 87us (506us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826393.070979:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24254 00000100:00100000:2.0:1787826393.071014:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d98ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346376320:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1787826393.262344:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826394.284334:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826395.308714:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826396.334572:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826396.379880:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.379927:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.379967:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.379990:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.380015:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.380039:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.380071:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.380098:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.380167:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826396.380206:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332712448 00000100:00100000:2.0:1787826396.380219:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332712448 00000100:00100000:2.0:1787826396.380290:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332712832 00000100:00100000:2.0:1787826396.380294:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332712832 00000100:00100000:2.0:1787826396.380314:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332712320 00000100:00100000:2.0:1787826396.380317:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332712320 00000100:00100000:2.0:1787826396.380338:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332712960 00000100:00100000:2.0:1787826396.380342:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332712960 00000100:00100000:2.0:1787826396.380370:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14439 00000100:00100000:2.0:1787826396.380382:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332712448:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826396.380473:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332712448:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 93us (545us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826396.380483:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14439 00000100:00100000:2.0:1787826396.380503:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14440 00000100:00100000:2.0:1787826396.380510:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332712832:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826396.380537:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848379f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2395:x1874663332712832:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (570us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826396.380544:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14440 00000100:00100000:2.0:1787826396.380558:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14441 00000100:00100000:2.0:1787826396.380564:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332712320:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826396.380588:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848378380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332712320:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (597us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826396.380594:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14441 00000100:00100000:2.0:1787826396.380605:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14442 00000100:00100000:2.0:1787826396.380611:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d98ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332712960:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826396.380634:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d98ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2395:x1874663332712960:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (595us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826396.380640:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14442 00000100:00100000:2.0:1787826396.380667:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332712576 00000100:00100000:2.0:1787826396.380671:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332712576 00000100:00100000:2.0:1787826396.380704:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332712704 00000100:00100000:2.0:1787826396.380708:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332712704 00000100:00100000:2.0:1787826396.380730:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332713088 00000100:00100000:2.0:1787826396.380733:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332713088 00000100:00100000:2.0:1787826396.380751:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332713216 00000100:00100000:2.0:1787826396.380754:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332713216 00000100:00100000:2.0:1787826396.380776:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8928 00000100:00100000:2.0:1787826396.380783:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332712576:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826396.380814:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784837b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332712576:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (797us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826396.380820:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8928 00000100:00100000:2.0:1787826396.380833:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30906 00000100:00100000:2.0:1787826396.380839:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2392:x1874663332712704:12345-192.168.203.56@tcp:9:kworker.0 00000020:00000080:2.0:1787826396.380862:0:8444:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xe21309aa81f4ad5d 00000100:00080000:2.0:1787826396.380872:0:8444:0:(import.c:65:import_set_state_nolock()) ffff9f784407f800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1787826396.380930:0:8444:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294976175 00000100:00100000:2.0:1787826396.381138:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+2:2392:x1874663332712704:12345-192.168.203.56@tcp:9:kworker.0 Request processed in 297us (1065us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826396.381145:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30906 00000100:00100000:2.0:1787826396.381159:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30907 00000100:00100000:2.0:1787826396.381165:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796eda4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2392:x1874663332713088:12345-192.168.203.56@tcp:9:kworker.0 00000020:00000080:2.0:1787826396.381179:0:8444:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xe21309aa81edfe1c 00000100:00080000:2.0:1787826396.381186:0:8444:0:(import.c:65:import_set_state_nolock()) ffff9f7844878000 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1787826396.381211:0:8444:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294973868 00000100:00100000:2.0:1787826396.381300:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796eda4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+2:2392:x1874663332713088:12345-192.168.203.56@tcp:9:kworker.0 Request processed in 134us (1200us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826396.381307:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30907 00000100:00100000:2.0:1787826396.381533:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30908 00000100:00100000:2.0:1787826396.381542:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2392:x1874663332713216:12345-192.168.203.56@tcp:9:kworker.0 00000020:00000080:2.0:1787826396.381554:0:8444:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xe21309aa81edf7f5 00000100:00080000:2.0:1787826396.381561:0:8444:0:(import.c:65:import_set_state_nolock()) ffff9f7946763800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1787826396.381583:0:8444:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294976175 00000100:00100000:2.0:1787826396.381641:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+2:2392:x1874663332713216:12345-192.168.203.56@tcp:9:kworker.0 Request processed in 100us (1473us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826396.381648:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30908 00000100:00080000:2.0:1787826396.381691:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826396.381696:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826396.381699:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826396.381702:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1787826396.381952:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826396.382985:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332712192 00000100:00100000:3.0:1787826396.383002:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332712192 00000100:00100000:3.0:1787826396.383085:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2254 00000100:00100000:3.0:1787826396.383099:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332712192:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826396.383204:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2395:x1874663332712192:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 109us (3340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826396.383213:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2254 00010000:00010000:3.0:1787826396.383251:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826396.383256:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1787826396.397446:0:8438:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of fc0b407b-a541-420d-8712-b9c8da18fe8c 00000020:00000080:2.0:1787826396.397461:0:8438:0:(genops.c:885:class_export_put()) final put ffff9f784487f800/fc0b407b-a541-420d-8712-b9c8da18fe8c 00000001:00080000:2.0:1787826396.397587:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973868 is committed 00000020:00000080:3.0:1787826396.397743:0:156572:0:(genops.c:844:class_export_destroy()) destroying export ffff9f784487f800/fc0b407b-a541-420d-8712-b9c8da18fe8c for lustre-OST0000 00000020:00000080:3.0:1787826396.397890:0:156572:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9f7844878000 for lustre-OST0000 00000100:00100000:0.0:1787826397.164662:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346376832:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.164759:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826397.164837:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346377344:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.164864:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826397.164888:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346377856:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.164926:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826397.164964:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1787826397.164970:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3655:1874663346376704:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.164997:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826397.165015:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346377216:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165035:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826397.165064:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346377728:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165089:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826397.165158:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346376832 00000100:00100000:0.0:1787826397.165169:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346376832 00000100:00100000:0.0:1787826397.165234:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346377344 00000100:00100000:0.0:1787826397.165238:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346377344 00000100:00100000:0.0:1787826397.165258:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346376704 00000100:00100000:0.0:1787826397.165261:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346376704 00000100:00100000:0.0:1787826397.165279:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346377216 00000100:00100000:0.0:1787826397.165282:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346377216 00000100:00100000:0.0:1787826397.165303:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14443 00000100:00100000:0.0:1787826397.165313:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843436d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346376832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165377:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843436d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346376832:12345-0@lo:400:kworker.0 Request processed in 64us (618us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826397.165385:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14443 00000100:00100000:0.0:1787826397.165407:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14444 00000100:00100000:0.0:1787826397.165414:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843434380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346377344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165443:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843434380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346377344:12345-0@lo:400:kworker.0 Request processed in 29us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826397.165450:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14444 00000100:00100000:0.0:1787826397.165463:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14445 00000100:00100000:0.0:1787826397.165469:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3655:x1874663346376704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165493:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848303800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3655:x1874663346376704:12345-0@lo:400:kworker.0 Request processed in 25us (495us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826397.165499:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14445 00000100:00100000:0.0:1787826397.165510:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14446 00000100:00100000:0.0:1787826397.165516:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346377216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165540:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848300380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346377216:12345-0@lo:400:kworker.0 Request processed in 25us (504us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826397.165546:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14446 00000100:00100000:0.0:1787826397.165609:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3655:1874663346376704:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165620:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346377216:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165629:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346376832:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.165639:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346377344:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.165805:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346376448:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.165845:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826397.165887:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346376960:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.165912:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826397.165931:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346377472:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.165952:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826397.165968:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346377984:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.165987:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826397.166006:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1787826397.166013:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346376576:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166040:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826397.166073:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346377088:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166096:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826397.166137:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346377600:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166159:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1787826397.166192:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826397.166197:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826397.166210:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346376960 00000100:00100000:2.0:1787826397.166215:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346376960 00000100:00100000:2.0:1787826397.166255:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346377984 00000100:00100000:2.0:1787826397.166258:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346377984 00000100:00100000:2.0:1787826397.166276:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14447 00000100:00100000:2.0:1787826397.166284:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346376960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166325:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346376960:12345-0@lo:400:kworker.0 Request processed in 42us (412us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826397.166331:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14447 00000100:00100000:2.0:1787826397.166347:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14448 00000100:00100000:2.0:1787826397.166353:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346377984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166382:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c96a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346377984:12345-0@lo:400:kworker.0 Request processed in 29us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826397.166388:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14448 00000100:00100000:2.0:1787826397.166409:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346377856 00000100:00100000:2.0:1787826397.166413:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346377856 00000100:00100000:2.0:1787826397.166440:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346377728 00000100:00100000:2.0:1787826397.166443:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346377728 00000100:00100000:2.0:1787826397.166462:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346377472 00000100:00100000:2.0:1787826397.166465:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346377472 00000100:00100000:2.0:1787826397.166484:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346377088 00000100:00100000:2.0:1787826397.166487:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346377088 00000100:00100000:2.0:1787826397.166508:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8929 00000100:00100000:2.0:1787826397.166514:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346377856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166555:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346377856:12345-0@lo:400:kworker.0 Request processed in 41us (1628us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826397.166562:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8929 00000100:00100000:2.0:1787826397.166575:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8930 00000100:00100000:2.0:1787826397.166581:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346377728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166608:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7848301880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346377728:12345-0@lo:400:kworker.0 Request processed in 28us (1520us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826397.166615:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8930 00000100:00100000:2.0:1787826397.166625:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8931 00000100:00100000:2.0:1787826397.166631:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346377472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166660:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346377472:12345-0@lo:400:kworker.0 Request processed in 29us (707us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826397.166667:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8931 00000100:00100000:2.0:1787826397.166679:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8932 00000100:00100000:2.0:1787826397.166685:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346377088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166727:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784cc30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346377088:12345-0@lo:400:kworker.0 Request processed in 43us (630us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826397.166737:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8932 00000100:00100000:2.0:1787826397.166766:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346376576 00000100:00100000:2.0:1787826397.166769:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346376576 00000100:00100000:2.0:1787826397.166797:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346377600 00000100:00100000:2.0:1787826397.166800:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346377600 00000100:00100000:2.0:1787826397.166814:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24255 00000100:00100000:2.0:1787826397.166819:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346376576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166853:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c94000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346376576:12345-0@lo:400:kworker.0 Request processed in 34us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826397.166858:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24255 00000100:00100000:2.0:1787826397.166869:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24256 00000100:00100000:2.0:1787826397.166876:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346377600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166901:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d815f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346377600:12345-0@lo:400:kworker.0 Request processed in 26us (741us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826397.166906:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24256 00000100:00080000:2.0:1787826397.166951:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826397.166954:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826397.166956:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787826397.166986:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346376576:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.166999:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346377088:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.167008:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346377600:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.167018:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346376960:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.167028:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346377472:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.167038:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346377984:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.167100:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973868 gen 2 00000004:00080000:2.0:1787826397.167136:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7842c93100, transno 4294973868 00000100:00100000:2.0:1787826397.167146:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346377728:0@lo:400:kworker.0 00000100:00100000:2.0:1787826397.167160:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346377856:0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.167946:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346376448 00000100:00100000:0.0:1787826397.167958:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346376448 00000100:00100000:0.0:1787826397.168014:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2255 00000100:00100000:0.0:1787826397.168024:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346376448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826397.168140:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346376448:12345-0@lo:400:kworker.0 Request processed in 118us (2294us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826397.168148:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2255 00000100:00100000:2.0:1787826397.168342:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346376448:0@lo:400:kworker.0 00010000:00010000:0.0:1787826397.358813:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826397.676564:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346378112:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826397.676660:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826397.676802:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346378112 00000100:00100000:0.0:1787826397.676813:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346378112 00000100:00100000:0.0:1787826397.676885:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6291 00000100:00100000:0.0:1787826397.676897:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346378112:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826397.676949:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 207424 pending 0 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826397.680707:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c321f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346378112:12345-0@lo:13:osp-pre-1-1.0 Request processed in 3795us (4030us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826397.680726:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6291 00000100:00100000:3.0:1787826397.681371:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843643b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346378112:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826398.188318:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787826398.188356:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346378240:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826398.188453:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826398.188513:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346378240 00000100:00100000:2.0:1787826398.188523:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346378240 00000100:00100000:2.0:1787826398.188577:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24257 00000100:00100000:2.0:1787826398.188587:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346378240:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826398.188629:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:2.0:1787826398.188679:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346378240:12345-0@lo:41:osp-pre-1-0.0 Request processed in 92us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826398.188686:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24257 00000100:00100000:2.0:1787826398.188751:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f684a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346378240:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826398.189194:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826398.189208:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346378368:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826398.189245:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826398.189293:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346378368 00000100:00100000:0.0:1787826398.189298:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346378368 00000100:00100000:0.0:1787826398.189339:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6292 00000100:00100000:0.0:1787826398.189347:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346378368:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787826398.189368:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 207424 pending 0 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826398.189408:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843643b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346378368:12345-0@lo:13:osp-pre-1-0.0 Request processed in 62us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826398.189414:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6292 00000100:00100000:0.0:1787826398.189466:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346378368:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826398.190824:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346378496:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826398.190888:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826398.190910:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346378624:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826398.190945:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826398.191372:0:125377:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346378496 00000100:00100000:1.0:1787826398.191382:0:125377:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346378496 00000100:00100000:1.0:1787826398.191453:0:125377:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24258 00000100:00100000:1.0:1787826398.191465:0:125377:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346378496:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826398.191504:0:125377:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:1.0:1787826398.191569:0:125377:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346378496:12345-0@lo:41:osp-pre-0-1.0 Request processed in 105us (680us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826398.191577:0:125377:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24258 00000100:00100000:1.0:1787826398.191628:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346378496:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826398.194539:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346378624 00000100:00100000:1.0:1787826398.194553:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346378624 00000100:00100000:1.0:1787826398.194633:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6293 00000100:00100000:1.0:1787826398.194647:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346378624:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826398.194684:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:3.0:1787826398.194833:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826398.194852:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346378752:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826398.194944:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826398.196396:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346378752 00000100:00100000:0.0:1787826398.196414:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346378752 00000100:00100000:0.0:1787826398.196470:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6294 00000100:00100000:0.0:1787826398.196481:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346378752:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787826398.196511:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826398.196620:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346378752:12345-0@lo:13:osp-pre-0-1.0 Request processed in 140us (1675us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826398.196632:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6294 00000100:00100000:2.0:1787826398.197177:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843642a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346378752:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826398.197925:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346378624:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3276us (6973us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826398.197935:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6293 00000100:00100000:2.0:1787826398.198166:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f797c320000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346378624:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1787826398.381274:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826398.708986:0:168345:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 46b: pdirops: link vs create ======================================================================== 06:26:36 (1787826396) 00010000:00010000:3.0:1787826399.409818:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826400.358546:0:168393:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:3.0:1787826400.358593:0:168393:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:0.0:1787826400.430763:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826401.453953:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826401.512334:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.512385:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.512422:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.512452:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.512469:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.512491:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.512518:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332713472 00000100:00100000:2.0:1787826401.512527:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332713472 00000100:00100000:2.0:1787826401.512599:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332713984 00000100:00100000:2.0:1787826401.512604:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332713984 00000100:00100000:2.0:1787826401.512624:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332713600 00000100:00100000:2.0:1787826401.512627:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332713600 00000100:00100000:2.0:1787826401.512645:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332713856 00000100:00100000:2.0:1787826401.512648:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332713856 00000100:00100000:2.0:1787826401.512670:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14449 00000100:00100000:2.0:1787826401.512681:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332713472:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826401.512759:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332713472:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 80us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826401.512766:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14449 00000100:00100000:2.0:1787826401.512788:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14450 00000100:00100000:2.0:1787826401.512796:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332713984:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826401.512824:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f684a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332713984:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 29us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826401.512831:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14450 00000100:00100000:2.0:1787826401.512843:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14451 00000100:00100000:2.0:1787826401.512849:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332713600:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826401.512874:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797cf2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332713600:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (403us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826401.512880:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14451 00000100:00100000:2.0:1787826401.512894:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14452 00000100:00100000:2.0:1787826401.512900:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332713856:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826401.512923:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332713856:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826401.512935:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14452 00000100:00080000:2.0:1787826401.512953:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826401.512956:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826401.514900:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332713728 00000100:00100000:3.0:1787826401.514914:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332713728 00000100:00100000:3.0:1787826401.514994:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8933 00000100:00100000:3.0:1787826401.515007:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332713728:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826401.515093:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797c320000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332713728:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 89us (2706us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826401.515101:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8933 00000100:00100000:3.0:1787826401.515145:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332713344 00000100:00100000:3.0:1787826401.515150:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332713344 00000100:00100000:3.0:1787826401.515176:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2256 00000100:00100000:3.0:1787826401.515183:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784c3c6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332713344:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826401.515205:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784c3c6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332713344:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (2752us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826401.515212:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2256 00000100:00080000:0.0:1787826401.515384:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826401.722878:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826401.723016:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332714112 00000100:00100000:0.0:1787826401.723035:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332714112 00000100:00100000:0.0:1787826401.723132:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68749 00000100:00100000:0.0:1787826401.723144:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:228391:x1874663332714112:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:0.0:1787826401.723165:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826401.723189:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cade lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d16 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.723258:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cae5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.723271:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cae5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.723282:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cae5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826401.723327:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826401.723331:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826401.723352:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cae5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.723378:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cade lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d16 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.723395:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cae5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d16 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.723404:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826401.723407:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826401.723410:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826401.723474:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b75aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:228391:x1874663332714112:12345-192.168.203.56@tcp:101:touch.0 Request processed in 332us (604us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826401.723481:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68749 00000100:00100000:0.0:1787826401.732666:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.733923:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332714368 00000100:00100000:2.0:1787826401.733936:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332714368 00000100:00100000:2.0:1787826401.734015:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68750 00000100:00100000:2.0:1787826401.734036:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:228391:x1874663332714368:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:2.0:1787826401.734057:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826401.734060:0:35094:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826401.734089:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cae5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d16 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.734131:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826401.734137:0:35094:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826401.734150:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4caec lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d1d expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.734266:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.734280:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.734290:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.734304:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.734311:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.734319:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826401.735253:0:35094:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x525:0x0], index 1: 0 00010000:00010000:2.0:1787826401.735422:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cb01 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735435:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cb01 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735445:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cb01 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735481:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735489:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735495:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735502:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735508:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490eea00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826401.735512:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735519:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735535:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826401.735537:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826401.735541:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735547:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4caf3 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826401.735559:0:35094:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000422009a7 reply state 000000005641a037 transno 8589957518 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735569:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735574:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd54468e8 (ffff9f784abfe700) 00010000:00010000:2.0:1787826401.735576:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826401.735578:0:35094:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826401.735610:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eec00/0xe21309aa81f4caec lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d1d expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735624:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cb01 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d1d expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826401.735632:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f784abfe800) 00010000:00010000:2.0:1787826401.735634:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826401.735636:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:2.0:1787826401.735717:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843642a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:228391:x1874663332714368:12345-192.168.203.56@tcp:101:touch.0 Request processed in 1683us (3060us total) trans 8589957518 rc 0/0 00000100:00100000:2.0:1787826401.735725:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68750 00000100:00100000:2.0:1787826401.743399:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826401.745402:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332714624 00000100:00100000:1.0:1787826401.745419:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332714624 00000100:00100000:1.0:1787826401.745509:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68751 00000100:00100000:1.0:1787826401.745521:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:228391:x1874663332714624:12345-192.168.203.56@tcp:36:touch.0 00010000:00010000:1.0:1787826401.745596:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787826401.745627:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cb01 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d1d expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826401.745648:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f784abfe800) 00010000:00010000:1.0:1787826401.745660:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826401.745716:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826401.745728:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826401.745737:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826401.745848:0:6528:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ebe48f21 reply state 00000000f13d7417 transno 8589957519 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826401.745867:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826401.745873:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:1.0:1787826401.745876:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826401.745878:0:6528:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787826401.745993:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:228391:x1874663332714624:12345-192.168.203.56@tcp:36:touch.0 Request processed in 473us (2603us total) trans 8589957519 rc 0/0 00000100:00100000:1.0:1787826401.746002:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68751 00000100:00100000:1.0:1787826401.753209:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826401.753438:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332714752 00000100:00100000:0.0:1787826401.753448:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332714752 00000100:00100000:0.0:1787826401.753522:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68752 00000100:00100000:0.0:1787826401.753533:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:228391:x1874663332714752:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:0.0:1787826401.753557:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826401.753581:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cb16 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d24 expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.753638:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cb16 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6e9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d24 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.753653:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cb16 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6e9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d24 expref: 11 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826401.753671:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:0.0:1787826401.753675:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1787826401.753679:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42c00, rc 0) 00000100:00100000:0.0:1787826401.753739:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:228391:x1874663332714752:12345-192.168.203.56@tcp:101:touch.0 Request processed in 209us (537us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826401.753747:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68752 00000100:00100000:2.0:1787826401.763329:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826401.763412:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332714880 00000100:00100000:3.0:1787826401.763423:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332714880 00000100:00100000:3.0:1787826401.763472:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30909 00000100:00100000:3.0:1787826401.763481:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2392:x1874663332714880:12345-192.168.203.56@tcp:8:touch.0 00010000:00080000:3.0:1787826401.763515:0:8444:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from 398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp t0 exp 0000000000000000 cur 8213 last 0 00000020:00000080:3.0:1787826401.763546:0:8444:0:(genops.c:1385:class_connect()) connect: client 398e0287-d3c7-44dd-b1d9-6b85f698e615, cookie 0xe21309aa81f4cb1d 00002000:00100000:3.0:1787826401.763557:0:8444:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli 398e0287-d3c7-44dd-b1d9-6b85f698e615/ffff9f784487f800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113950 ocd_grant: 8437760 ocd_index: 0 ocd_group 0 00002000:00100000:3.0:1787826401.763576:0:8444:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:3.0:1787826401.763653:0:8444:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:3.0:1787826401.763683:0:8444:0:(import.c:65:import_set_state_nolock()) ffff9f7844878000 : changing import state from RECOVER to FULL 00000100:00100000:3.0:1787826401.763742:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2392:x1874663332714880:12345-192.168.203.56@tcp:8:touch.0 Request processed in 262us (427us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826401.763750:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30909 00000001:00100000:2.0:1787826401.767534:0:9524:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000001:00100000:2.0:1787826401.767552:0:9524:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fc0b407b-a541-420d-8712-b9c8da18fe8c 00000020:00000080:2.0:1787826401.767556:0:9524:0:(genops.c:885:class_export_put()) final put ffff9f7946760000/fc0b407b-a541-420d-8712-b9c8da18fe8c 00000001:00100000:2.0:1787826401.767585:0:9524:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:2.0:1787826401.767587:0:9524:0:(genops.c:885:class_export_put()) final put ffff9f784407e000/398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:3.0:1787826401.767625:0:21470:0:(genops.c:844:class_export_destroy()) destroying export ffff9f7946760000/fc0b407b-a541-420d-8712-b9c8da18fe8c for lustre-OST0001 00000020:00000080:3.0:1787826401.767702:0:21470:0:(genops.c:844:class_export_destroy()) destroying export ffff9f784407e000/398e0287-d3c7-44dd-b1d9-6b85f698e615 for lustre-OST0001 00000020:00000080:3.0:1787826401.767722:0:21470:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9f7946763800 for lustre-OST0001 00000020:00000080:3.0:1787826401.767727:0:21470:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9f784407f800 for lustre-OST0001 00000100:00100000:0.0:1787826401.767976:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.768069:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332715008 00000100:00100000:2.0:1787826401.768078:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332715008 00000100:00100000:2.0:1787826401.768179:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30910 00000100:00100000:2.0:1787826401.768191:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:0:x1874663332715008:12345-192.168.203.56@tcp:2:touch.0 00000100:00100000:2.0:1787826401.768718:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332715008:12345-192.168.203.56@tcp:2:touch.0 Request processed in 254us (471us total) trans 4294976176 rc 0/0 00000100:00100000:2.0:1787826401.768728:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30910 00000100:00080000:3.0:1787826401.769017:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826401.774873:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826401.775749:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332715264 00000100:00100000:2.0:1787826401.775760:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332715264 00000100:00100000:2.0:1787826401.775836:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9410 00000100:00100000:2.0:1787826401.775847:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:228391:x1874663332715264:12345-192.168.203.56@tcp:35:touch.0 00000100:00100000:2.0:1787826401.776748:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:228391:x1874663332715264:12345-192.168.203.56@tcp:35:touch.0 Request processed in 896us (1877us total) trans 8589957520 rc 0/0 00000100:00100000:2.0:1787826401.776758:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9410 00000100:00100000:1.0:1787826402.286082:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346379264:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286193:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286253:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346379776:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286278:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286302:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346380288:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286328:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286355:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:1.0:1787826402.286361:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346378880:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286388:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286409:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346379392:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286430:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286446:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346379904:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286469:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286484:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346380416:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286505:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286530:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346379008:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286556:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286571:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346379520:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286591:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826402.286598:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346379264 00000100:00100000:3.0:1787826402.286606:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346379264 00000100:00100000:1.0:1787826402.286607:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346380032:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286628:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.286643:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1787826402.286649:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346379136:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.286659:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346379776 00000100:00100000:3.0:1787826402.286662:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346379776 00000100:00100000:3.0:1787826402.286679:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346379392 00000100:00100000:1.0:1787826402.286681:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826402.286682:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346379392 00000100:00100000:1.0:1787826402.286696:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346379648:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.286699:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346380416 00000100:00100000:3.0:1787826402.286701:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346380416 00000100:00100000:3.0:1787826402.286715:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346379136 00000100:00100000:1.0:1787826402.286716:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826402.286717:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346379136 00000100:00100000:3.0:1787826402.286731:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346379648 00000100:00100000:1.0:1787826402.286732:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346380160:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.286733:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346379648 00000100:00100000:1.0:1787826402.286751:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826402.286754:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14453 00000100:00100000:3.0:1787826402.286763:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850358700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346379264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286779:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346378880 00000100:00100000:1.0:1787826402.286782:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346378880 00000100:00100000:1.0:1787826402.286813:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2257 00000100:00100000:3.0:1787826402.286814:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850358700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346379264:12345-0@lo:400:kworker.0 Request processed in 53us (622us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826402.286819:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f797dfa4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346378880:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.286821:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14453 00000100:00100000:3.0:1787826402.286841:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14454 00000100:00100000:3.0:1787826402.286847:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346379776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286848:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f797dfa4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346378880:12345-0@lo:400:kworker.0 Request processed in 30us (460us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826402.286854:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2257 00000100:00100000:3.0:1787826402.286871:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785035ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346379776:12345-0@lo:400:kworker.0 Request processed in 25us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826402.286876:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14454 00000100:00100000:1.0:1787826402.286880:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346380288 00000100:00100000:1.0:1787826402.286883:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346380288 00000100:00100000:3.0:1787826402.286888:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14455 00000100:00100000:3.0:1787826402.286893:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346379392:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286907:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346379904 00000100:00100000:1.0:1787826402.286910:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346379904 00000100:00100000:3.0:1787826402.286917:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b618380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0000_UUID+5:3654:x1874663346379392:12345-0@lo:400:kworker.0 Request processed in 24us (487us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826402.286923:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14455 00000100:00100000:1.0:1787826402.286929:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346379520 00000100:00100000:1.0:1787826402.286932:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346379520 00000100:00100000:3.0:1787826402.286933:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14456 00000100:00100000:3.0:1787826402.286937:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346380416:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.286949:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346380160 00000100:00100000:1.0:1787826402.286951:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346380160 00000100:00100000:3.0:1787826402.286961:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3654:x1874663346380416:12345-0@lo:400:kworker.0 Request processed in 23us (455us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826402.286967:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14456 00000100:00100000:1.0:1787826402.286970:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8934 00000100:00100000:1.0:1787826402.286975:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346380288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.286978:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14457 00000100:00100000:3.0:1787826402.286983:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346379136:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.287003:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346380288:12345-0@lo:400:kworker.0 Request processed in 28us (674us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826402.287008:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346379136:12345-0@lo:400:kworker.0 Request processed in 26us (328us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826402.287009:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8934 00000100:00100000:3.0:1787826402.287014:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14457 00000100:00100000:1.0:1787826402.287040:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8935 00000100:00100000:3.0:1787826402.287040:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14458 00000100:00100000:1.0:1787826402.287047:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346379904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.287047:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346379648:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.287074:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b61ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3654:x1874663346379904:12345-0@lo:400:kworker.0 Request processed in 29us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826402.287074:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346379648:12345-0@lo:400:kworker.0 Request processed in 28us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826402.287080:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8935 00000100:00100000:3.0:1787826402.287080:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14458 00000100:00100000:1.0:1787826402.287091:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8936 00000100:00100000:1.0:1787826402.287097:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346379520:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1787826402.287131:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826402.287135:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826402.287137:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826402.287139:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826402.287143:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346379520:12345-0@lo:400:kworker.0 Request processed in 47us (552us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826402.287148:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8936 00000100:00100000:1.0:1787826402.287159:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8937 00000100:00100000:1.0:1787826402.287166:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346380160:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.287191:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346380160:12345-0@lo:400:kworker.0 Request processed in 26us (439us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826402.287196:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8937 00000100:00100000:1.0:1787826402.287239:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346379136:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.287248:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346379648:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.287260:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346380160:0@lo:400:kworker.0 00000100:00100000:1.0:1787826402.287268:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346379520:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.287272:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346378880:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.287282:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3654:1874663346379392:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.287289:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346379904:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.287298:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3654:1874663346380416:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.287308:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346379264:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.287316:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346379776:0@lo:400:kworker.0 00000100:00100000:3.0:1787826402.287324:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346380288:0@lo:400:kworker.0 00000100:00100000:2.0:1787826402.287642:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346379008 00000100:00100000:2.0:1787826402.287647:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346379008 00000100:00100000:2.0:1787826402.287681:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346380032 00000100:00100000:2.0:1787826402.287683:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346380032 00000100:00100000:2.0:1787826402.287708:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24259 00000100:00100000:2.0:1787826402.287715:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346379008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826402.287771:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f785124e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346379008:12345-0@lo:400:kworker.0 Request processed in 56us (1213us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826402.287777:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24259 00000100:00100000:2.0:1787826402.287792:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24260 00000100:00100000:2.0:1787826402.287808:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346380032:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826402.287836:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346380032:12345-0@lo:400:kworker.0 Request processed in 38us (1207us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826402.287841:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24260 00000100:00100000:0.0:1787826402.287980:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346379008:0@lo:400:kworker.0 00000100:00100000:0.0:1787826402.287991:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346380032:0@lo:400:kworker.0 00010000:00010000:0.0:1787826402.476516:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826402.797762:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346380544:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826402.797913:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826402.801101:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346380544 00000100:00100000:1.0:1787826402.801145:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346380544 00000100:00100000:1.0:1787826402.801229:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6295 00000100:00100000:1.0:1787826402.801242:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346380544:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826402.801285:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826402.801393:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346380544:12345-0@lo:13:osp-pre-1-1.0 Request processed in 153us (3481us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826402.801401:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6295 00000100:00100000:3.0:1787826402.801609:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346380544:0@lo:13:osp-pre-1-1.0 00000001:00080000:1.0:1787826402.803077:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957520 is committed 00010000:00010000:3.0:1787826402.803170:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803187:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803196:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803202:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826402.803208:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803216:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803291:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd54468e8 (ffff9f784abfe700) 00010000:00010000:3.0:1787826402.803296:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826402.803302:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803309:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4cafa lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803342:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803349:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803355:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803361:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826402.803364:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803370:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803469:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826402.803472:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826402.803476:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826402.803481:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cb0f lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826403.309737:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346380672:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826403.309824:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826403.309886:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346380800:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826403.309914:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826403.309937:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787826403.309943:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346380928:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826403.309964:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826403.309997:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346380672 00000100:00100000:2.0:1787826403.310006:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346380672 00000100:00100000:2.0:1787826403.310070:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346380928 00000100:00100000:2.0:1787826403.310074:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346380928 00000100:00100000:2.0:1787826403.310095:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24261 00000100:00100000:2.0:1787826403.310122:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346380672:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826403.310165:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:2.0:1787826403.310215:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346380672:12345-0@lo:41:osp-pre-0-1.0 Request processed in 109us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826403.310222:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24261 00000100:00100000:2.0:1787826403.310242:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24262 00000100:00100000:2.0:1787826403.310249:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346380928:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826403.310268:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:2.0:1787826403.310298:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346380928:12345-0@lo:41:osp-pre-1-0.0 Request processed in 51us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826403.310305:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24262 00000100:00100000:2.0:1787826403.310325:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346380800 00000100:00100000:2.0:1787826403.310329:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346380800 00000100:00100000:2.0:1787826403.310355:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6296 00000100:00100000:2.0:1787826403.310361:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346380800:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826403.310382:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826403.310410:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346380800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 49us (495us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826403.310416:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6296 00000100:00100000:2.0:1787826403.310482:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346380928:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826403.310496:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346380800:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826403.310527:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346380672:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826403.311594:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346381056:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826403.311650:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826403.311943:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346381184:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826403.311988:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826403.312597:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346381056 00000100:00100000:1.0:1787826403.312607:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346381056 00000100:00100000:1.0:1787826403.312656:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346381184 00000100:00100000:1.0:1787826403.312660:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346381184 00000100:00100000:1.0:1787826403.312681:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6297 00000100:00100000:1.0:1787826403.312692:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346381056:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826403.312719:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826403.312776:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346381056:12345-0@lo:13:osp-pre-0-1.0 Request processed in 85us (1125us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826403.312783:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6297 00000100:00100000:1.0:1787826403.312802:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6298 00000100:00100000:1.0:1787826403.312808:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346381184:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826403.312821:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826403.312857:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346381184:12345-0@lo:13:osp-pre-1-0.0 Request processed in 49us (868us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826403.312863:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6298 00000100:00100000:1.0:1787826403.312913:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346381184:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826403.314153:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78435b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346381056:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826403.314582:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826403.314652:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332715392 00000100:00100000:1.0:1787826403.314663:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332715392 00000100:00100000:1.0:1787826403.314752:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68753 00000100:00100000:1.0:1787826403.314765:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:228454:x1874663332715392:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:1.0:1787826403.314796:0:35092:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826403.314827:0:35092:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cb24 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d2b expref: -99 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826403.314911:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4cb2b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826403.314930:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4cb2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826403.314943:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4cb2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826403.315080:0:35092:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826403.315085:0:35092:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826403.315146:0:35092:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4cb2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826403.315177:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cb24 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d2b expref: 9 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826403.315204:0:35092:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4cb2b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d2b expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826403.315214:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826403.315218:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826403.315222:0:35092:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece000, rc 0) 00000100:00100000:1.0:1787826403.315289:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:228454:x1874663332715392:12345-192.168.203.56@tcp:101:link.0 Request processed in 527us (710us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826403.315297:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68753 00000100:00080000:3.0:1787826403.317216:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826403.319917:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826403.319972:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332715520 00000100:00100000:3.0:1787826403.319984:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332715520 00000100:00100000:3.0:1787826403.320063:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68754 00000100:00100000:3.0:1787826403.320077:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+8:228454:x1874663332715520:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:3.0:1787826403.320135:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826403.320162:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cb32 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d32 expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.320860:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.320878:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.320891:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.320908:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.320916:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.320923:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321158:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb47 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321168:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb47 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321175:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb47 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826403.321220:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6e9:0x0]: returning size 0 00000004:00200000:3.0:1787826403.321230:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826403.321251:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb47 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321266:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb47 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321275:0:35093:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321282:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321288:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321295:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321300:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826403.321305:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321313:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321327:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826403.321330:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826403.321336:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321342:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cb39 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321362:0:35093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321368:0:35093:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321374:0:35093:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321379:0:35093:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826403.321382:0:35093:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321396:0:35093:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321407:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd54468e8 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826403.321409:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826403.321412:0:35093:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321417:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cb40 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321454:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cb32 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d32 expref: 10 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321469:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb47 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d32 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.321483:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826403.321485:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826403.321489:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68200, rc 0) 00000100:00100000:3.0:1787826403.321572:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:228454:x1874663332715520:12345-192.168.203.56@tcp:101:link.0 Request processed in 1496us (1668us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826403.321581:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68754 00000100:00100000:1.0:1787826403.324520:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826403.324842:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332715776 00000100:00100000:3.0:1787826403.324856:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332715776 00000100:00100000:3.0:1787826403.324967:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68755 00000100:00100000:3.0:1787826403.324981:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f796cd7f850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:228454:x1874663332715776:12345-192.168.203.56@tcp:36:link.0 00010000:00010000:3.0:1787826403.325064:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1787826403.325101:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb47 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d32 expref: 9 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.325158:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826403.325171:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4cb2b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d2b expref: 8 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.325181:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826403.325184:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826403.325214:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.325230:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.325248:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.325265:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.325272:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826403.325279:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00020000:3.0:1787826403.325287:0:6528:0:(mdt_handler.c:4171:mdt_object_pdo_lock()) cfs_fail_timeout id 145 sleeping for 15000ms 00010000:00010000:3.0:1787826403.500246:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826403.873829:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826403.874039:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332715904 00000100:00100000:0.0:1787826403.874055:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332715904 00000100:00100000:0.0:1787826403.874219:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68756 00000100:00100000:0.0:1787826403.874234:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:228456:x1874663332715904:12345-192.168.203.56@tcp:101:multiop.0 00010000:00010000:0.0:1787826403.874267:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:3.0:1787826403.874904:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787826403.874960:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cb5c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d39 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826403.875059:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826403.875075:0:24703:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 2/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826403.875090:0:24703:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826403.875138:0:24703:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826403.875152:0:24703:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826404.533809:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826405.560078:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00020000:3.0:1787826405.829157:0:6528:0:(mdt_handler.c:4171:mdt_object_pdo_lock()) cfs_fail_timeout interrupted 00010000:00010000:3.0:1787826405.829209:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829236:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829253:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826405.829642:0:6528:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b03cd77a reply state 00000000636b125f transno 8589957521 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829658:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829673:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826405.829676:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00000100:00100000:3.0:1787826405.829681:0:6528:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826405.829688:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829696:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829703:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829712:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829718:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826405.829724:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829732:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829744:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826405.829746:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826405.829750:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829758:0:6528:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829765:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829816:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829823:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cb4e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826405.829836:0:6528:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b03cd77a reply state 00000000636b125f transno 8589957521 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829845:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826405.829851:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa8c465fb (ffff9f79460a2600) 00010000:00010000:3.0:1787826405.829854:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826405.829856:0:6528:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:3.0:1787826405.829966:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f796cd7f850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:228454:x1874663332715776:12345-192.168.203.56@tcp:36:link.0 Request processed in 2504983us (2505453us total) trans 8589957521 rc 0/0 00000100:00100000:3.0:1787826405.829976:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68755 00010000:00010000:1.0:1787826405.832581:0:24703:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.832711:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826405.832859:0:24703:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826405.832864:0:24703:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826405.832894:0:24703:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.832944:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cb5c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d39 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.832970:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d39 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.832981:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826405.832984:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826405.832988:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42600, rc 0) 00000100:00100000:1.0:1787826405.833126:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:228456:x1874663332715904:12345-192.168.203.56@tcp:101:multiop.0 Request processed in 1958871us (1959286us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826405.833137:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68756 00000100:00100000:2.0:1787826405.845916:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826405.846401:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332716160 00000100:00100000:1.0:1787826405.846415:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332716160 00000100:00100000:1.0:1787826405.846580:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68757 00000100:00100000:1.0:1787826405.846594:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:228456:x1874663332716160:12345-192.168.203.56@tcp:101:multiop.0 00010000:00010000:1.0:1787826405.846692:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826405.846696:0:11629:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787826405.846730:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb63 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d39 expref: 7 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.846753:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826405.846759:0:11629:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826405.846776:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cb71 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d40 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847029:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847043:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847054:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847066:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847074:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847341:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847412:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847422:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847429:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826405.847566:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6e9:0x0]: returning size 0 00010000:00010000:1.0:1787826405.847586:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847594:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826405.847713:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787826405.847859:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847866:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826405.847871:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847880:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847899:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:1.0:1787826405.847902:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826405.847906:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847911:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cb86 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847932:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847938:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847943:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847949:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847953:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826405.847956:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847962:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847969:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826405.847972:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826405.847974:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.847983:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4cb78 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848001:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848007:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848012:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848016:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecfc00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826405.848019:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848024:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848030:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa8c465fb (ffff9f79460a2600) 00010000:00010000:1.0:1787826405.848032:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826405.848035:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848039:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cb7f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848074:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cb71 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d40 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848385:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826405.848388:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826405.848393:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cb71 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d40 expref: 7 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826405.848404:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826405.849024:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:228456:x1874663332716160:12345-192.168.203.56@tcp:101:multiop.0 Request processed in 2426us (3114us total) trans 0 rc 301/301 00000100:00100000:1.0:1787826405.849038:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68757 00000100:00100000:2.0:1787826406.390493:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826406.390668:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332716288 00000100:00100000:0.0:1787826406.390678:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332716288 00000100:00100000:0.0:1787826406.390755:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68758 00000100:00100000:0.0:1787826406.390766:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:228140:x1874663332716288:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826406.390791:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826406.390814:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb8d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d47 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826406.390876:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cb94 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826406.390890:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cb94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826406.390901:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cb94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826406.391346:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826406.391351:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826406.391378:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cb94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826406.391408:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cb8d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d47 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826406.391428:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cb94 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d47 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826406.391437:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826406.391440:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826406.391444:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42a00, rc 0) 00000100:00080000:3.0:1787826406.391635:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826406.391672:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:228140:x1874663332716288:12345-192.168.203.56@tcp:101:bash.0 Request processed in 906us (1186us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826406.391680:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68758 00000100:00100000:2.0:1787826406.404154:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826406.404268:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332716416 00000100:00100000:0.0:1787826406.404280:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332716416 00000100:00100000:0.0:1787826406.405467:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9411 00000100:00100000:0.0:1787826406.405486:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7967aee050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:228140:x1874663332716416:12345-192.168.203.56@tcp:37:bash.0 00000100:00100000:1.0:1787826406.412381:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7967aee050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:228140:x1874663332716416:12345-192.168.203.56@tcp:37:bash.0 Request processed in 6891us (8225us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826406.412395:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9411 00000100:00100000:1.0:1787826406.439317:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826406.439580:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332716544 00000100:00100000:2.0:1787826406.439590:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332716544 00000100:00100000:2.0:1787826406.439662:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68759 00000100:00100000:2.0:1787826406.439673:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:228498:x1874663332716544:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826406.439695:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826406.439717:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4cb9b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d4e expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.439785:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.439800:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.439812:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.439823:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.439831:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.439839:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440530:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cbb0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440545:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cbb0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440554:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cbb0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826406.440599:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6e9:0x0]: returning size 0 00000004:00200000:3.0:1787826406.440604:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826406.440623:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cbb0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440635:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cbb0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440644:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440651:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440656:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440663:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440669:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826406.440673:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440681:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440692:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826406.440695:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826406.440698:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440704:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cba2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440725:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440731:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440736:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440741:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efe00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826406.440744:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440750:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440757:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa8c465fb (ffff9f79460a2600) 00010000:00010000:3.0:1787826406.440760:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826406.440763:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440769:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cba9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440802:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4cb9b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d4e expref: 12 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440815:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cbb0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d4e expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.440827:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826406.440829:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826406.440832:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69800, rc 0) 00000100:00100000:3.0:1787826406.441087:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:228498:x1874663332716544:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1414us (1779us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826406.441096:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68759 00000100:00100000:3.0:1787826406.444618:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826406.444725:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332716672 00000100:00100000:1.0:1787826406.444736:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332716672 00000100:00100000:1.0:1787826406.444812:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30911 00000100:00100000:1.0:1787826406.444824:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332716672:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826406.444842:0:35097:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826406.444867:0:35097:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d55 expref: -99 pid: 35097 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826406.445005:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:1.0:1787826406.445054:0:35097:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x525:0x0] initial LVB size: 0, mtime: 0x6a9010e0, atime: 0x6a9010e0, ctime: 0x6a9010e0, blocks: 0x0 00010000:00010000:1.0:1787826406.445075:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x525:0x0].0x0 since=8218s v=(0 0) 00010000:00010000:1.0:1787826406.445080:0:35097:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x525:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787826406.445090:0:35097:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 7 pid: 35097 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.447712:0:35097:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 7 pid: 35097 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.447736:0:35097:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826406.447814:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2394:x1874663332716672:12345-192.168.203.56@tcp:101:rm.0 Request processed in 2992us (3204us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826406.447823:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30911 00000100:00100000:0.0:1787826406.455878:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826406.456177:0:35092:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332716928 00000100:00100000:1.0:1787826406.456187:0:35092:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332716928 00000100:00100000:1.0:1787826406.456250:0:35092:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68760 00000100:00100000:1.0:1787826406.456260:0:35092:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:228498:x1874663332716928:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:1.0:1787826406.456316:0:35092:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1787826406.456476:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cbb0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d4e expref: 11 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456503:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cb16 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6e9:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d24 expref: 10 pid: 35092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456515:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:1.0:1787826406.456525:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cb94 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d47 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456534:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826406.456537:0:35092:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826406.456561:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456574:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456584:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456594:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456602:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456609:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456699:0:35092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456708:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.456715:0:35092:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457056:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457065:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457072:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457078:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecec00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826406.457083:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457090:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457098:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:1.0:1787826406.457101:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826406.457311:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457320:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cbcc lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457348:0:35092:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457354:0:35092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457360:0:35092:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457366:0:35092:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457371:0:35092:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826406.457374:0:35092:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457383:0:35092:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457390:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826406.457393:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826406.457396:0:35092:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457401:0:35092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cbbe lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826406.457410:0:35092:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000461a5ac7 reply state 00000000965c87c7 transno 8589957522 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457421:0:35092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826406.457427:0:35092:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa8c465fb (ffff9f79460a2600) 00010000:00010000:1.0:1787826406.457430:0:35092:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826406.457432:0:35092:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787826406.457563:0:35092:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_009:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:228498:x1874663332716928:12345-192.168.203.56@tcp:36:rm.0 Request processed in 1300us (1692us total) trans 8589957522 rc 0/0 00000100:00100000:1.0:1787826406.457571:0:35092:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68760 00000100:00100000:2.0:1787826406.462819:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826406.462955:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332717056 00000100:00100000:3.0:1787826406.462965:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332717056 00000100:00100000:3.0:1787826406.463173:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68761 00000100:00100000:3.0:1787826406.463185:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:228498:x1874663332717056:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826406.463209:0:35093:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826406.463231:0:35093:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cbd3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d5c expref: -99 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.463865:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cbda lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.463882:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cbda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.463894:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cbda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826406.463944:0:35093:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826406.463948:0:35093:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826406.463971:0:35093:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cbda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.464014:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cbd3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d5c expref: 12 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.464035:0:35093:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cbda lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d5c expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.464043:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826406.464046:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826406.464049:0:35093:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69200, rc 0) 00000100:00100000:3.0:1787826406.464154:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:228498:x1874663332717056:12345-192.168.203.56@tcp:101:rm.0 Request processed in 971us (1343us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826406.464162:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68761 00000100:00100000:3.0:1787826406.475848:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826406.476917:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332717184 00000100:00100000:1.0:1787826406.476929:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332717184 00000100:00100000:2.0:1787826406.477401:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68762 00000100:00100000:2.0:1787826406.477415:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:228498:x1874663332717184:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826406.477436:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826406.477457:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4cbe1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d63 expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477535:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477548:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477559:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477574:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477582:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477588:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477843:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cbf6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477854:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cbf6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477861:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cbf6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826406.477903:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6e9:0x0]: returning size 0 00000004:00200000:2.0:1787826406.477908:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826406.477926:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cbf6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477939:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cbf6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477953:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477960:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477966:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477972:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.477977:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826406.477981:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478000:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478012:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826406.478015:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826406.478019:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478025:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4cbe8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478038:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478043:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478049:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478053:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826406.478056:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478061:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478067:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd54468e8 (ffff9f784abfed00) 00010000:00010000:2.0:1787826406.478070:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826406.478073:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478079:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4cbef lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478142:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4cbe1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d63 expref: 13 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478157:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cbf6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d63 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826406.478163:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:2.0:1787826406.478165:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826406.478168:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826406.478253:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:228498:x1874663332717184:12345-192.168.203.56@tcp:101:rm.0 Request processed in 830us (2404us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826406.478269:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68762 00000100:00100000:0.0:1787826406.483213:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826406.483691:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332717440 00000100:00100000:3.0:1787826406.483703:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332717440 00000100:00100000:3.0:1787826406.483766:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68763 00000100:00100000:3.0:1787826406.483776:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:228498:x1874663332717440:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826406.483834:0:6528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1787826406.483866:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cbf6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6e9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d63 expref: 12 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.483888:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826406.483900:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cbda lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d5c expref: 11 pid: 35093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.483915:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826406.483918:0:6528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826406.483942:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.483956:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.483965:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.483981:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.483998:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484006:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484101:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484131:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484138:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787826406.484716:0:6528:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].218, 64 off22144 00000004:00080000:3.0:1787826406.484727:0:6528:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8218696620758ns, next at 8219696618618ns, rc = 0 00010000:00010000:3.0:1787826406.484770:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484780:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010080000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484788:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010080000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484795:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010080000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484802:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484808:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x50010480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484829:0:6528:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826406.484846:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484852:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484858:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484863:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826406.484868:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484875:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484884:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826406.484886:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826406.484890:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484895:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc0b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6e9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484911:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484917:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484922:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484929:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484934:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826406.484937:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484942:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484948:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826406.484950:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826406.484953:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484963:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cbfd lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826406.484972:0:6528:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005e13a040 reply state 00000000decb7709 transno 8589957523 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484981:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.484998:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd54468e8 (ffff9f79460a2e00) 00010000:00010000:3.0:1787826406.485000:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826406.485003:0:6528:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787826406.485074:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:228498:x1874663332717440:12345-192.168.203.56@tcp:36:rm.0 Request processed in 1297us (1868us total) trans 8589957523 rc 0/0 00000100:00100000:3.0:1787826406.485082:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68763 00010000:00010000:3.0:1787826406.485140:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826406.485145:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.485152:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68e00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826406.485156:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.485164:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.485173:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:3.0:1787826406.485175:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787826406.485178:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.485188:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc12 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6e9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826406.485197:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826406.485198:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826406.485200:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826406.572785:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826406.624015:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826406.624067:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826406.624148:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826406.624177:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826406.624386:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332717568 00000100:00100000:0.0:1787826406.624396:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332717568 00000100:00100000:0.0:1787826406.624460:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2258 00000100:00100000:0.0:1787826406.624470:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d60c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332717568:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826406.624545:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d60c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332717568:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 77us (540us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826406.624557:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2258 00000100:00100000:1.0:1787826406.624755:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332717824 00000100:00100000:1.0:1787826406.624761:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332717824 00000100:00100000:1.0:1787826406.624820:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8938 00000100:00100000:1.0:1787826406.624829:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332717824:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826406.624868:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332717824:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 41us (801us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826406.624875:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8938 00000100:00100000:1.0:1787826406.624895:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332718080 00000100:00100000:1.0:1787826406.624899:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332718080 00000100:00100000:1.0:1787826406.624923:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332717696 00000100:00100000:1.0:1787826406.624926:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332717696 00000100:00100000:1.0:1787826406.624945:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14459 00000100:00100000:1.0:1787826406.624951:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332718080:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826406.624979:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332718080:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (873us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826406.624998:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14459 00000100:00100000:1.0:1787826406.625011:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14460 00000100:00100000:1.0:1787826406.625017:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332717696:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826406.625043:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332717696:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 27us (865us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826406.625049:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14460 00000100:00080000:0.0:1787826406.625135:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787826406.625140:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826406.626923:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826406.632415:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332717952 00000100:00100000:0.0:1787826406.632430:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332717952 00000100:00100000:0.0:1787826406.632511:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14461 00000100:00100000:0.0:1787826406.632520:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332717952:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826406.632613:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2394:x1874663332717952:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 94us (5699us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826406.632620:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14461 00000100:00100000:3.0:1787826407.405404:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346381312:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.405490:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826407.405562:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787826407.405569:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346381440:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.405601:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826407.405647:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346381568:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.405676:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826407.405716:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346381568 00000100:00100000:3.0:1787826407.405723:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346381568 00000100:00100000:3.0:1787826407.405775:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14462 00000100:00100000:3.0:1787826407.405785:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346381568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.405840:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346381568:12345-0@lo:400:kworker.0 Request processed in 57us (163us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826407.405847:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14462 00000100:00100000:3.0:1787826407.405888:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346381568:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.405923:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346381440 00000100:00100000:3.0:1787826407.405926:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346381440 00000100:00100000:3.0:1787826407.405952:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24263 00000100:00100000:3.0:1787826407.405958:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78435b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346381440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.406007:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78435b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346381440:12345-0@lo:400:kworker.0 Request processed in 49us (405us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826407.406014:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24263 00000100:00100000:3.0:1787826407.406037:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346381440:0@lo:400:kworker.0 00000100:00100000:1.0:1787826407.406226:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346381312 00000100:00100000:1.0:1787826407.406232:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346381312 00000100:00100000:1.0:1787826407.406273:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2259 00000100:00100000:1.0:1787826407.406281:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d63450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346381312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826407.406323:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d63450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3654:x1874663346381312:12345-0@lo:400:kworker.0 Request processed in 42us (833us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826407.406330:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2259 00000100:00100000:3.0:1787826407.406588:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d8bca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4305192b-43a6-4bf4-9f25-68faf720ed51:3654:1874663346381312:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.408602:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346382080:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.408676:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826407.408719:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787826407.408725:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346381696:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.408750:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826407.408772:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843091180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346382208:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.408794:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826407.408830:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346382080 00000100:00100000:2.0:1787826407.408837:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346382080 00000100:00100000:2.0:1787826407.408892:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346381696 00000100:00100000:2.0:1787826407.408896:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346381696 00000100:00100000:2.0:1787826407.408918:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346382208 00000100:00100000:2.0:1787826407.408921:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346382208 00000100:00100000:2.0:1787826407.408941:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14463 00000100:00100000:2.0:1787826407.408949:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346382080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409015:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8be680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346382080:12345-0@lo:400:kworker.0 Request processed in 66us (340us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826407.409023:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14463 00000100:00100000:2.0:1787826407.409045:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14464 00000100:00100000:2.0:1787826407.409052:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346381696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409080:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346381696:12345-0@lo:400:kworker.0 Request processed in 28us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826407.409086:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14464 00000100:00100000:2.0:1787826407.409097:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14465 00000100:00100000:2.0:1787826407.409200:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346382208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409233:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346382208:12345-0@lo:400:kworker.0 Request processed in 130us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826407.409240:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14465 00000100:00080000:2.0:1787826407.409387:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826407.409392:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826407.409429:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346381696:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409440:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843091180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346382208:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409449:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346382080:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409585:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843090700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346381952:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409621:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826407.409646:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1787826407.409653:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843090e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346381824:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409675:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826407.409695:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346382336:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409717:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826407.409752:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346381952 00000100:00100000:2.0:1787826407.409756:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346381952 00000100:00100000:2.0:1787826407.409797:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346382336 00000100:00100000:2.0:1787826407.409800:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346382336 00000100:00100000:2.0:1787826407.409822:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8939 00000100:00100000:2.0:1787826407.409830:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346381952:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409868:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d8bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346381952:12345-0@lo:400:kworker.0 Request processed in 39us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826407.409874:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8939 00000100:00100000:2.0:1787826407.409889:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8940 00000100:00100000:2.0:1787826407.409896:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346382336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.409923:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346382336:12345-0@lo:400:kworker.0 Request processed in 28us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826407.409929:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8940 00000100:00100000:2.0:1787826407.409946:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346381824 00000100:00100000:2.0:1787826407.409950:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346381824 00000100:00100000:2.0:1787826407.409977:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14466 00000100:00100000:2.0:1787826407.410014:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346381824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.410056:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346381824:12345-0@lo:400:kworker.0 Request processed in 44us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826407.410063:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14466 00000100:00080000:2.0:1787826407.410081:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826407.410085:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826407.410087:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787826407.410142:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843090e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346381824:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.410153:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346382336:0@lo:400:kworker.0 00000100:00100000:2.0:1787826407.410162:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843090700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346381952:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.410434:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843092300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346382464:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.410469:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826407.410494:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826407.410584:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843090700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346382848:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.410614:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826407.410645:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843092a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346382592:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.410669:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826407.410689:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826407.410694:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346382720:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.410717:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826407.410748:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346382464 00000100:00100000:3.0:1787826407.410753:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346382464 00000100:00100000:3.0:1787826407.410789:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24264 00000100:00100000:3.0:1787826407.410797:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346382464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.410839:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346382464:12345-0@lo:400:kworker.0 Request processed in 43us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826407.410847:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24264 00000100:00100000:3.0:1787826407.410869:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346382848 00000100:00100000:3.0:1787826407.410872:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346382848 00000100:00100000:3.0:1787826407.410897:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14467 00000100:00100000:3.0:1787826407.410903:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346382848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.410934:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346382848:12345-0@lo:400:kworker.0 Request processed in 32us (320us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826407.410941:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14467 00000100:00100000:3.0:1787826407.410962:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346382592 00000100:00100000:3.0:1787826407.410965:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346382592 00000100:00100000:3.0:1787826407.411013:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346382720 00000100:00100000:3.0:1787826407.411016:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346382720 00000100:00100000:3.0:1787826407.411042:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8941 00000100:00100000:3.0:1787826407.411049:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346382592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.411085:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346382592:12345-0@lo:400:kworker.0 Request processed in 37us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826407.411091:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8941 00000100:00100000:3.0:1787826407.411127:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8942 00000100:00100000:3.0:1787826407.411135:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346382720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.411163:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346382720:12345-0@lo:400:kworker.0 Request processed in 30us (445us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826407.411168:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8942 00000100:00100000:3.0:1787826407.413637:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843090700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346382848:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.413658:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843092300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346382464:0@lo:400:kworker.0 00000100:00080000:3.0:1787826407.413705:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826407.413713:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826407.413745:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843093480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346382720:0@lo:400:kworker.0 00000100:00100000:3.0:1787826407.413758:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7843092a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346382592:0@lo:400:kworker.0 00010000:00010000:0.0:1787826407.596286:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826407.916699:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346382976:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826407.916794:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826407.917049:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346382976 00000100:00100000:1.0:1787826407.917058:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346382976 00000100:00100000:1.0:1787826407.917143:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6299 00000100:00100000:1.0:1787826407.917153:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346382976:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826407.917183:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826407.917240:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346382976:12345-0@lo:13:osp-pre-1-1.0 Request processed in 89us (447us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826407.917247:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6299 00000100:00100000:3.0:1787826407.917778:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433efb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346382976:0@lo:13:osp-pre-1-1.0 00000001:00080000:0.0:1787826407.921069:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976176 is committed 00000004:00080000:1.0:1787826408.428386:0:30914:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8220640276220ns, next at 8221640257528ns, rc = 0 00000100:00100000:3.0:1787826408.430193:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346383104:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826408.430296:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826408.430552:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346383104 00000100:00100000:2.0:1787826408.430552:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346383232:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826408.430562:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346383104 00000100:00100000:2.0:1787826408.430594:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826408.430623:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346383232 00000100:00100000:1.0:1787826408.430628:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346383232 00000100:00100000:1.0:1787826408.430664:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6300 00000100:00100000:1.0:1787826408.430673:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346383104:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826408.430697:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826408.430744:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346383104:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (449us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826408.430750:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6300 00000100:00100000:1.0:1787826408.430766:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6301 00000100:00100000:1.0:1787826408.430772:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346383232:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787826408.430789:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:3.0:1787826408.430846:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346383360:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826408.430881:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826408.430948:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346383232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 174us (354us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826408.430955:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6301 00000100:00100000:1.0:1787826408.431030:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346383488:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826408.431155:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346383232:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826408.431179:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826408.431186:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346383616:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826408.431210:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826408.431367:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346383104:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826408.431420:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346383616 00000100:00100000:1.0:1787826408.431425:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346383616 00000100:00100000:1.0:1787826408.431494:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6302 00000100:00100000:1.0:1787826408.431504:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346383616:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826408.431517:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826408.431566:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346383616:12345-0@lo:13:osp-pre-0-1.0 Request processed in 64us (355us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826408.431573:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6302 00000100:00100000:3.0:1787826408.431591:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346383616:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826408.431651:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826408.432584:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346383360 00000100:00100000:3.0:1787826408.432594:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346383360 00000100:00100000:3.0:1787826408.432672:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346383488 00000100:00100000:3.0:1787826408.432676:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346383488 00000100:00100000:2.0:1787826408.432727:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24265 00000100:00100000:2.0:1787826408.432738:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346383360:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826408.432769:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:2.0:1787826408.432837:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843092300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346383360:12345-0@lo:41:osp-pre-1-0.0 Request processed in 102us (1955us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826408.432844:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24265 00000100:00100000:2.0:1787826408.432863:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24266 00000100:00100000:2.0:1787826408.432871:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346383488:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826408.432889:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:2.0:1787826408.432921:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346383488:12345-0@lo:41:osp-pre-0-1.0 Request processed in 51us (1270us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826408.432926:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24266 00000100:00100000:1.0:1787826408.433026:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346383360:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826408.433059:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346383488:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1787826408.623835:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826409.646346:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826410.668268:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1787826411.511612:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 224 committed 00000001:00080000:0.0:1787826411.511654:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957523 is committed 00000004:00080000:0.0:1787826411.511659:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 223 committed 00010000:00010000:2.0:1787826411.511781:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511794:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511803:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511810:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826411.511815:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511823:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826411.511838:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346383744:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1787826411.511845:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa8c465fb (ffff9f79460a2600) 00010000:00010000:2.0:1787826411.511848:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826411.511853:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511859:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68a00/0xe21309aa81f4cb55 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511871:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511876:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511881:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511885:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826411.511888:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511893:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511899:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6e9:0x0].0x0 (ffff9f7961614400) 00010000:00010000:2.0:1787826411.511901:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00010000:00010000:2.0:1787826411.511904:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511910:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68200/0xe21309aa81f4cb6a lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x6e9:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826411.511915:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787826411.511939:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511946:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511960:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511965:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ecee00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826411.511968:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511974:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511981:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa8c465fb (ffff9f79460a2600) 00010000:00010000:2.0:1787826411.511983:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826411.511986:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.511992:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cbc5 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xa8c465fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.512007:0:3638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.512014:0:3638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.512032:0:3638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.512038:0:3638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826411.512041:0:3638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.512048:0:3638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.512056:0:3638:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd54468e8 (ffff9f79460a2e00) 00010000:00010000:2.0:1787826411.512058:0:3638:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826411.512060:0:3638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.512065:0:3638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69800/0xe21309aa81f4cc04 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xd54468e8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826411.512543:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346383744 00000100:00100000:0.0:1787826411.512551:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346383744 00000100:00100000:0.0:1787826411.512620:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30912 00000100:00100000:0.0:1787826411.512630:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346383744:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1787826411.512658:0:24218:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:1317 count 1 00010000:00010000:0.0:1787826411.512683:0:24218:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826411.512703:0:24218:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 5 pid: 35097 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826411.512712:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x525:0x0].0x0 since=5s v=(0 0) 00010000:00010000:0.0:1787826411.512717:0:24218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x525:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826411.512720:0:24218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826411.512750:0:24218:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 5 pid: 35097 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826411.512764:0:24218:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 5 pid: 35097 timeout: 8323 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826411.512776:0:24218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784e5c9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:24218:1874663346383872:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826411.512842:0:24218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f7848e219c0 going to sleep for 11 seconds 00000100:00080000:2.0:1787826411.512894:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826411.521092:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826411.521425:0:8459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332718208 00000100:00100000:3.0:1787826411.521436:0:8459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332718208 00010000:00010000:3.0:1787826411.521505:0:8459:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 6 pid: 35097 timeout: 8323 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826411.521548:0:8459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22094 00000100:00100000:3.0:1787826411.521558:0:8459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332718208:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826411.521572:0:8459:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826411.521577:0:8459:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 5/0,0 mode: PR/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 6 pid: 35097 timeout: 8323 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826411.521593:0:8459:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 6 pid: 35097 timeout: 8323 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826411.521620:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x525:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826411.521624:0:8459:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x525:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826411.521630:0:8459:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826411.521639:0:8459:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826411.521658:0:8459:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826411.521714:0:8459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332718208:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 158us (632us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826411.521722:0:8459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22094 00000100:00080000:2.0:1787826411.522029:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787826411.522404:0:24218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ae0f1415/0xe21309aa81f4cbb7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d55 expref: 5 pid: 35097 timeout: 8323 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826411.522420:0:24218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784e5c9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:24218:1874663346383872:192.168.203.56@tcp:104: 00010000:00010000:1.0:1787826411.522442:0:24218:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826411.522452:0:24218:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826411.522458:0:24218:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826411.522467:0:24218:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826411.522481:0:24218:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826411.523308:0:24218:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826411.523525:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346383744:12345-0@lo:6:osp-syn-1-0.0 Request processed in 10894us (11607us total) trans 4294976177 rc 0/0 00000100:00100000:1.0:1787826411.523535:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30912 00000004:00080000:2.0:1787826411.524157:0:3653:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f784a11ea00/2, rc 0, transno 4294976177 00000100:00100000:2.0:1787826411.524166:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346383744:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1787826411.527382:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826411.527400:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826411.527416:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826411.527422:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826411.527435:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826411.527465:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826411.527473:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a57c3a21/0xe21309aa81f4cc19 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 24218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826411.529316:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826411.529324:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826411.529327:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826411.692970:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826411.752903:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826411.752934:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826411.752981:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826411.753008:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826411.753031:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826411.753058:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332718336 00000100:00100000:0.0:1787826411.753067:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332718336 00000100:00100000:0.0:1787826411.753146:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2260 00000100:00100000:0.0:1787826411.753157:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d63050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332718336:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826411.753322:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332718464 00000100:00100000:1.0:1787826411.753326:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332718464 00000100:00100000:1.0:1787826411.753371:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332718848 00000100:00100000:1.0:1787826411.753374:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332718848 00000100:00100000:1.0:1787826411.753393:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332718592 00000100:00100000:1.0:1787826411.753396:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332718592 00000100:00100000:1.0:1787826411.753413:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332718976 00000100:00100000:1.0:1787826411.753416:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332718976 00000100:00100000:1.0:1787826411.753444:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14468 00000100:00100000:1.0:1787826411.753453:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332718464:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826411.753518:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d63050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2396:x1874663332718336:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 363us (534us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826411.753527:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2260 00000100:00100000:3.0:1787826411.753623:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14469 00000100:00100000:3.0:1787826411.753634:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332718848:12345-192.168.203.56@tcp:400:kworker.0 00000100:00080000:2.0:1787826411.757418:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826411.757427:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787826411.757470:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14470 00000100:00100000:2.0:1787826411.757490:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332718592:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826411.757759:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2396:x1874663332718464:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 4304us (4862us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826411.757769:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14468 00000100:00100000:1.0:1787826411.757798:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14471 00000100:00100000:1.0:1787826411.757808:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784e5cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332718976:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826411.757838:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2396:x1874663332718976:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 32us (4805us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826411.757844:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14471 00000100:00100000:3.0:1787826411.772421:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2396:x1874663332718848:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 18776us (19471us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826411.772440:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14469 00000100:00100000:3.0:1787826411.772534:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826411.772817:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332718720 00000100:00100000:3.0:1787826411.772826:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332718720 00000100:00100000:3.0:1787826411.772880:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30913 00000100:00100000:3.0:1787826411.772890:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2392:x1874663332718720:12345-192.168.203.56@tcp:9:kworker.0 00000020:00000080:3.0:1787826411.772917:0:17826:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xe21309aa81f4c9f7 00000100:00080000:3.0:1787826411.772927:0:17826:0:(import.c:65:import_set_state_nolock()) ffff9f78435b8000 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1787826411.772984:0:17826:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294973868 00000100:00100000:3.0:1787826411.773163:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+2:2392:x1874663332718720:12345-192.168.203.56@tcp:9:kworker.0 Request processed in 274us (628us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826411.773171:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30913 00000100:00080000:3.0:1787826411.773188:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1787826411.773201:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826411.776252:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784e5c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2396:x1874663332718592:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 18758us (23228us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826411.776268:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14470 00000100:00100000:2.0:1787826412.524800:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346384384:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.524878:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826412.524921:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346384896:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.524957:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826412.524976:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346385408:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525003:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826412.525031:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1787826412.525037:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346384256:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525057:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826412.525073:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346384768:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525094:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826412.525138:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346385280:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525160:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826412.525192:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346384384 00000100:00100000:2.0:1787826412.525200:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346384384 00000100:00100000:2.0:1787826412.525265:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346384896 00000100:00100000:2.0:1787826412.525269:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346384896 00000100:00100000:2.0:1787826412.525288:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346384256 00000100:00100000:2.0:1787826412.525291:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346384256 00000100:00100000:2.0:1787826412.525318:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346384768 00000100:00100000:2.0:1787826412.525321:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346384768 00000100:00100000:2.0:1787826412.525341:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14472 00000100:00100000:2.0:1787826412.525350:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346384384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525411:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874663346384384:12345-0@lo:400:kworker.0 Request processed in 61us (533us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826412.525422:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14472 00000100:00100000:2.0:1787826412.525441:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14473 00000100:00100000:2.0:1787826412.525446:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346384896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525474:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874663346384896:12345-0@lo:400:kworker.0 Request processed in 28us (530us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826412.525479:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14473 00000100:00100000:2.0:1787826412.525491:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14474 00000100:00100000:2.0:1787826412.525496:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346384256:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525523:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874663346384256:12345-0@lo:400:kworker.0 Request processed in 27us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826412.525529:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14474 00000100:00100000:2.0:1787826412.525541:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14475 00000100:00100000:2.0:1787826412.525546:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346384768:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525573:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874663346384768:12345-0@lo:400:kworker.0 Request processed in 26us (477us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826412.525578:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14475 00000100:00100000:2.0:1787826412.525604:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346385408 00000100:00100000:2.0:1787826412.525607:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346385408 00000100:00100000:2.0:1787826412.525629:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346385280 00000100:00100000:2.0:1787826412.525631:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346385280 00000100:00100000:2.0:1787826412.525648:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8943 00000100:00100000:2.0:1787826412.525654:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346385408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525684:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346385408:12345-0@lo:400:kworker.0 Request processed in 29us (679us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826412.525689:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8943 00000100:00100000:2.0:1787826412.525700:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8944 00000100:00100000:2.0:1787826412.525706:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346385280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525733:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346385280:12345-0@lo:400:kworker.0 Request processed in 27us (572us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826412.525738:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8944 00000100:00100000:2.0:1787826412.525796:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874663346384256:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525807:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3653:1874663346384768:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525815:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346385280:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525824:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3653:1874663346384384:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525832:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3653:1874663346384896:0@lo:400:kworker.0 00000100:00100000:2.0:1787826412.525841:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c93100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346385408:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526188:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346384128:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526240:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826412.526269:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346384640:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526302:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826412.526324:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346385152:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526346:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826412.526367:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:3.0:1787826412.526374:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346384000:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526405:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826412.526430:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346384512:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526452:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826412.526469:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346385024:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526488:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826412.526506:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346385536:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526526:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826412.526553:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346384512 00000100:00100000:3.0:1787826412.526558:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346384512 00000100:00100000:3.0:1787826412.526597:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346385536 00000100:00100000:3.0:1787826412.526612:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346385536 00000100:00100000:3.0:1787826412.526636:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14476 00000100:00100000:3.0:1787826412.526644:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346384512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526686:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346384512:12345-0@lo:400:kworker.0 Request processed in 42us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826412.526693:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14476 00000100:00100000:3.0:1787826412.526710:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14477 00000100:00100000:3.0:1787826412.526717:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346385536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526745:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346385536:12345-0@lo:400:kworker.0 Request processed in 29us (218us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826412.526751:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14477 00000100:00080000:3.0:1787826412.526773:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826412.526778:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826412.526779:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826412.526800:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346384640 00000100:00100000:3.0:1787826412.526804:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346384640 00000100:00100000:3.0:1787826412.526831:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346385024 00000100:00100000:3.0:1787826412.526834:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346385024 00000100:00100000:3.0:1787826412.526853:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8945 00000100:00100000:3.0:1787826412.526859:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346384640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526892:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346384640:12345-0@lo:400:kworker.0 Request processed in 33us (589us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826412.526899:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8945 00000100:00100000:3.0:1787826412.526912:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8946 00000100:00100000:3.0:1787826412.526924:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346385024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.526964:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346385024:12345-0@lo:400:kworker.0 Request processed in 46us (474us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826412.526970:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8946 00000100:00100000:3.0:1787826412.526999:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346384128 00000100:00100000:3.0:1787826412.527004:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346384128 00000100:00100000:3.0:1787826412.527027:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346385152 00000100:00100000:3.0:1787826412.527031:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346385152 00000100:00100000:3.0:1787826412.527048:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24267 00000100:00100000:3.0:1787826412.527054:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346384128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.527087:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7843093b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346384128:12345-0@lo:400:kworker.0 Request processed in 33us (846us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826412.527096:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24267 00000100:00100000:3.0:1787826412.527138:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24268 00000100:00100000:3.0:1787826412.527150:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346385152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.527181:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346385152:12345-0@lo:400:kworker.0 Request processed in 36us (834us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826412.527187:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24268 00000100:00080000:3.0:1787826412.527223:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1787826412.527226:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826412.527251:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346384512:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.527262:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346385024:0@lo:400:kworker.0 00000100:00100000:1.0:1787826412.527273:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346384000 00000100:00100000:3.0:1787826412.527274:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346385536:0@lo:400:kworker.0 00000100:00100000:1.0:1787826412.527281:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346384000 00000100:00100000:3.0:1787826412.527287:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346384128:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.527298:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346384640:0@lo:400:kworker.0 00000100:00100000:3.0:1787826412.527308:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346385152:0@lo:400:kworker.0 00000100:00100000:1.0:1787826412.527327:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2261 00000100:00100000:1.0:1787826412.527337:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d61850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346384000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826412.527390:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d61850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346384000:12345-0@lo:400:kworker.0 Request processed in 56us (984us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826412.527398:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2261 00000100:00100000:1.0:1787826412.527455:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784a11e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346384000:0@lo:400:kworker.0 00010000:00010000:3.0:1787826412.716304:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826413.036577:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346385664:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826413.036660:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826413.036775:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346385664 00000100:00100000:1.0:1787826413.036784:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346385664 00000100:00100000:1.0:1787826413.037035:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6303 00000100:00100000:1.0:1787826413.037047:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346385664:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826413.037077:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826413.037169:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346385664:12345-0@lo:13:osp-pre-1-1.0 Request processed in 125us (509us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826413.037177:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6303 00000100:00100000:0.0:1787826413.037289:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346385664:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826413.548349:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826413.548374:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346385792:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787826413.548446:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826413.548494:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346385920:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826413.548518:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826413.548578:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346385792 00000100:00100000:2.0:1787826413.548584:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346385792 00000100:00100000:2.0:1787826413.548645:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346385920 00000100:00100000:2.0:1787826413.548648:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346385920 00000100:00100000:2.0:1787826413.548668:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6304 00000100:00100000:2.0:1787826413.548675:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346385792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826413.548697:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826413.548741:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850772680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3653:x1874663346385792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 66us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826413.548748:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6304 00000100:00100000:2.0:1787826413.548767:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6305 00000100:00100000:2.0:1787826413.548772:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346385920:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1787826413.548789:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 207424 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826413.548815:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850773100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346385920:12345-0@lo:13:osp-pre-0-1.0 Request processed in 43us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826413.548820:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6305 00000100:00100000:2.0:1787826413.548840:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850771500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346386048:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826413.548870:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1787826413.548900:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1787826413.548918:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346386048 00000100:00100000:2.0:1787826413.548921:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346386048 00000100:00100000:2.0:1787826413.548960:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24269 00000100:00100000:2.0:1787826413.548967:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346386048:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826413.548990:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:2.0:1787826413.549030:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346386048:12345-0@lo:41:osp-pre-1-0.0 Request processed in 63us (157us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826413.549036:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24269 00000100:00100000:2.0:1787826413.549076:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346386048:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826413.549130:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346385920:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826413.549145:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850771880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346385792:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826413.551009:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7849dd0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346386176:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826413.551087:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826413.551136:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346386176 00000100:00100000:3.0:1787826413.551145:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346386176 00000100:00100000:3.0:1787826413.551191:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6306 00000100:00100000:3.0:1787826413.551200:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346386176:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1787826413.551221:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 207424 pending 0 free 3893981184 avail 3662458880 00000100:00100000:3.0:1787826413.551262:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433efb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346386176:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (184us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826413.551275:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6306 00000100:00100000:1.0:1787826413.553673:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346386304:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826413.553746:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826413.553966:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7849dd0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346386176:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787826413.554091:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346386304 00000100:00100000:1.0:1787826413.554098:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346386304 00000100:00100000:1.0:1787826413.554170:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24270 00000100:00100000:1.0:1787826413.554179:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346386304:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826413.554222:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:1.0:1787826413.554277:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346386304:12345-0@lo:41:osp-pre-0-1.0 Request processed in 99us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826413.554284:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24270 00000100:00100000:1.0:1787826413.554324:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f78433ec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346386304:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1787826413.743503:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787826413.893476:0:168735:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 46c: pdirops: link vs link ======================================================================== 06:26:51 (1787826411) 00010000:00010000:2.0:1787826414.764510:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826415.359035:0:168783:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:1.0:1787826415.359087:0:168783:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00010000:00010000:3.0:1787826415.788268:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826416.578822:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826416.578909:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332719104 00000100:00100000:2.0:1787826416.578930:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332719104 00000100:00100000:2.0:1787826416.579064:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68764 00000100:00100000:2.0:1787826416.579076:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:229181:x1874663332719104:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:2.0:1787826416.579097:0:35080:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826416.579138:0:35080:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4cc20 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d6a expref: -99 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.579209:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4cc27 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.579222:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4cc27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.579235:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4cc27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826416.579457:0:35080:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:2.0:1787826416.579462:0:35080:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826416.579487:0:35080:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4cc27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.579516:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4cc20 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d6a expref: 8 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.579533:0:35080:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4cc27 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d6a expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.579541:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826416.579544:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787826416.579548:0:35080:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef600, rc 0) 00000100:00100000:2.0:1787826416.579605:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:229181:x1874663332719104:12345-192.168.203.56@tcp:101:touch.0 Request processed in 532us (793us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826416.579613:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68764 00000100:00080000:0.0:1787826416.579997:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826416.592376:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826416.592562:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332719360 00000100:00100000:2.0:1787826416.592572:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332719360 00000100:00100000:2.0:1787826416.592629:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68765 00000100:00100000:2.0:1787826416.592639:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:229181:x1874663332719360:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:2.0:1787826416.592660:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826416.592663:0:11629:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787826416.592693:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef600/0xe21309aa81f4cc27 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d6a expref: 7 pid: 35080 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.592711:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826416.592716:0:11629:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826416.592730:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4cc2e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d71 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.592855:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.592868:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.592878:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.592893:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x73b310ba bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.592899:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.592905:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787826416.593311:0:11629:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x525:0x0], index 0: 0 00010000:00010000:2.0:1787826416.593464:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cc43 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593474:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cc43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593481:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cc43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593515:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593523:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593529:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593535:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593541:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efa00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826416.593546:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593553:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593566:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826416.593568:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826416.593571:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593577:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efa00/0xe21309aa81f4cc35 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826416.593588:0:11629:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cf8e315a reply state 000000001f68762a transno 8589957524 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593599:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593604:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x73b310ba (ffff9f784abfeb00) 00010000:00010000:2.0:1787826416.593607:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826416.593609:0:11629:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826416.593637:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efc00/0xe21309aa81f4cc2e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d71 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593650:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cc43 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d71 expref: 8 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826416.593657:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784abfe800) 00010000:00010000:2.0:1787826416.593659:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826416.593661:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef200, rc 0) 00000100:00100000:2.0:1787826416.593732:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7849dd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:229181:x1874663332719360:12345-192.168.203.56@tcp:101:touch.0 Request processed in 1094us (1366us total) trans 8589957524 rc 0/0 00000100:00100000:2.0:1787826416.593740:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68765 00000100:00100000:2.0:1787826416.607605:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826416.607706:0:35093:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332719616 00000100:00100000:0.0:1787826416.607717:0:35093:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332719616 00000100:00100000:0.0:1787826416.607785:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68766 00000100:00100000:0.0:1787826416.607796:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:229181:x1874663332719616:12345-192.168.203.56@tcp:36:touch.0 00010000:00010000:0.0:1787826416.607860:0:35093:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787826416.607892:0:35093:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef200/0xe21309aa81f4cc43 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d71 expref: 9 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826416.607914:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784abfe800) 00010000:00010000:0.0:1787826416.607937:0:35093:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826416.608011:0:35093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826416.608025:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826416.608035:0:35093:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826416.608186:0:35093:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000036e8154c reply state 0000000062fcfa62 transno 8589957525 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826416.608200:0:35093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826416.608211:0:35093:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:0.0:1787826416.608213:0:35093:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826416.608217:0:35093:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787826416.608323:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:229181:x1874663332719616:12345-192.168.203.56@tcp:36:touch.0 Request processed in 527us (727us total) trans 8589957525 rc 0/0 00000100:00100000:0.0:1787826416.608331:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68766 00000100:00100000:3.0:1787826416.613934:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826416.614017:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332719744 00000100:00100000:1.0:1787826416.614028:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332719744 00000100:00100000:1.0:1787826416.614097:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68767 00000100:00100000:1.0:1787826416.614135:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:229181:x1874663332719744:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:1.0:1787826416.614158:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826416.614180:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cc58 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d78 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826416.614263:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cc58 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6ea:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d78 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826416.614278:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cc58 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6ea:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d78 expref: 11 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826416.614297:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:1.0:1787826416.614301:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1787826416.614304:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfc00, rc 0) 00000100:00100000:1.0:1787826416.614367:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:229181:x1874663332719744:12345-192.168.203.56@tcp:101:touch.0 Request processed in 262us (456us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826416.614378:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68767 00000100:00100000:3.0:1787826416.618779:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826416.618875:0:17826:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332719872 00000100:00100000:1.0:1787826416.618886:0:17826:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332719872 00000100:00100000:1.0:1787826416.618941:0:17826:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30914 00000100:00100000:1.0:1787826416.618949:0:17826:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0+-99:2392:x1874663332719872:12345-192.168.203.56@tcp:8:touch.0 00010000:00080000:1.0:1787826416.618987:0:17826:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from 398e0287-d3c7-44dd-b1d9-6b85f698e615@192.168.203.56@tcp t4294972907 exp 0000000000000000 cur 8228 last 0 00000020:00000080:1.0:1787826416.619025:0:17826:0:(genops.c:1385:class_connect()) connect: client 398e0287-d3c7-44dd-b1d9-6b85f698e615, cookie 0xe21309aa81f4cc5f 00002000:00100000:1.0:1787826416.619038:0:17826:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli 398e0287-d3c7-44dd-b1d9-6b85f698e615/ffff9f7844d32000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113950 ocd_grant: 8351744 ocd_index: 0 ocd_group 0 00002000:00100000:1.0:1787826416.619056:0:17826:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:1.0:1787826416.619146:0:17826:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:1.0:1787826416.619176:0:17826:0:(import.c:65:import_set_state_nolock()) ffff9f7844d33000 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1787826416.619254:0:17826:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2392:x1874663332719872:12345-192.168.203.56@tcp:8:touch.0 Request processed in 303us (486us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826416.619261:0:17826:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30914 00000100:00100000:2.0:1787826416.637703:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826416.637788:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332720000 00000100:00100000:0.0:1787826416.637799:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332720000 00000100:00100000:0.0:1787826416.637876:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30915 00000100:00100000:0.0:1787826416.637889:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:0:x1874663332720000:12345-192.168.203.56@tcp:2:touch.0 00000100:00080000:2.0:1787826416.638304:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1787826416.641419:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973869 is committed 00000001:00100000:2.0:1787826416.641446:0:8438:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000001:00100000:2.0:1787826416.641457:0:8438:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:2.0:1787826416.641462:0:8438:0:(genops.c:885:class_export_put()) final put ffff9f78435be800/398e0287-d3c7-44dd-b1d9-6b85f698e615 00000020:00000080:3.0:1787826416.642611:0:21470:0:(genops.c:844:class_export_destroy()) destroying export ffff9f78435be800/398e0287-d3c7-44dd-b1d9-6b85f698e615 for lustre-OST0000 00000020:00000080:3.0:1787826416.642691:0:21470:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9f78435b8000 for lustre-OST0000 00000100:00100000:3.0:1787826416.643291:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:0:x1874663332720000:12345-192.168.203.56@tcp:2:touch.0 Request processed in 5398us (5591us total) trans 4294973869 rc 0/0 00000100:00100000:3.0:1787826416.643304:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30915 00000100:00100000:2.0:1787826416.652305:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826416.652362:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332720256 00000100:00100000:2.0:1787826416.652375:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332720256 00000100:00100000:2.0:1787826416.652453:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9412 00000100:00100000:2.0:1787826416.652467:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:229181:x1874663332720256:12345-192.168.203.56@tcp:35:touch.0 00000100:00100000:2.0:1787826416.652832:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:229181:x1874663332720256:12345-192.168.203.56@tcp:35:touch.0 Request processed in 367us (540us total) trans 8589957526 rc 0/0 00000100:00100000:2.0:1787826416.652841:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9412 00010000:00010000:3.0:1787826416.812364:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826416.866799:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826416.866854:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826416.866901:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826416.867022:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332720640 00000100:00100000:3.0:1787826416.867035:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332720640 00000100:00100000:3.0:1787826416.867166:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8947 00000100:00100000:3.0:1787826416.867190:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332720640:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826416.867287:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f78433ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332720640:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 102us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826416.867296:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8947 00000100:00080000:1.0:1787826416.867712:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826416.869587:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332720384 00000100:00100000:0.0:1787826416.869602:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332720384 00000100:00100000:0.0:1787826416.869687:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2262 00000100:00100000:0.0:1787826416.869699:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d60450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2393:x1874663332720384:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826416.869806:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d60450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2393:x1874663332720384:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 109us (3019us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826416.869814:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2262 00000100:00100000:2.0:1787826416.870079:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332720896 00000100:00100000:2.0:1787826416.870085:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332720896 00000100:00100000:2.0:1787826416.870159:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14478 00000100:00100000:2.0:1787826416.870169:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332720896:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826416.870214:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332720896:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 47us (3312us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826416.870221:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14478 00000100:00080000:1.0:1787826416.870293:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826416.876090:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826416.876193:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826416.876393:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332720512 00000100:00100000:1.0:1787826416.876404:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332720512 00000100:00100000:1.0:1787826416.876499:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332720768 00000100:00100000:1.0:1787826416.876503:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332720768 00000100:00100000:1.0:1787826416.876531:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14479 00000100:00100000:1.0:1787826416.876540:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332720512:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826416.876620:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332720512:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 80us (539us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826416.876628:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14479 00000100:00100000:1.0:1787826416.876646:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14480 00000100:00100000:1.0:1787826416.876653:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332720768:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826416.876676:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332720768:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (483us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826416.876682:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14480 00000100:00080000:2.0:1787826416.880441:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826416.880452:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826417.647500:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826417.647526:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346386816:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.647554:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3655:1874663346386688:0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.647626:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346386944:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.648020:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826417.648082:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346387200:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.648126:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826417.648151:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346387712:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.648401:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826417.648439:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:1.0:1787826417.648446:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346386560:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.648476:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826417.648496:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346387072:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.648517:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826417.648535:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346387584:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.648553:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826417.648583:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346386688 00000100:00100000:1.0:1787826417.648590:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346386688 00000100:00100000:1.0:1787826417.648637:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346387200 00000100:00100000:1.0:1787826417.648640:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346387200 00000100:00100000:3.0:1787826417.648692:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826417.648712:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346386432:0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.648746:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826417.648759:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826417.648788:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346387456:0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.648792:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346387328:0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.648814:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826417.648830:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826417.648836:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346387968:0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.648858:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346387840:0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.648865:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826417.648894:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826417.648942:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346386816 00000100:00100000:3.0:1787826417.648947:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346386816 00000100:00100000:0.0:1787826417.648998:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346387712 00000100:00100000:1.0:1787826417.648999:0:8752:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346386944 00000100:00100000:0.0:1787826417.649002:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346387712 00000100:00100000:1.0:1787826417.649002:0:8752:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346386944 00000100:00100000:2.0:1787826417.649046:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346387328 00000100:00100000:2.0:1787826417.649050:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346387328 00000100:00100000:3.0:1787826417.649057:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346387968 00000100:00100000:3.0:1787826417.649063:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346387968 00000100:00100000:2.0:1787826417.649090:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14481 00000100:00100000:3.0:1787826417.649102:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14482 00000100:00100000:2.0:1787826417.649122:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-MDT0001_UUID+5:3655:x1874663346386688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.649128:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346387200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.649183:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3655:x1874663346387200:12345-0@lo:400:kworker.0 Request processed in 58us (1056us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826417.649190:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14482 00000100:00100000:3.0:1787826417.649209:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14483 00000100:00100000:3.0:1787826417.649215:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346386816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.649249:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3652:x1874663346386816:12345-0@lo:400:kworker.0 Request processed in 34us (556us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826417.649255:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14483 00000100:00100000:3.0:1787826417.649268:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14484 00000100:00100000:3.0:1787826417.649274:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346387328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.649301:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977f29880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874663346387328:12345-0@lo:400:kworker.0 Request processed in 28us (473us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826417.649306:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14484 00000100:00100000:3.0:1787826417.649319:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14485 00000100:00100000:3.0:1787826417.649326:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346387968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.649357:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346387968:12345-0@lo:400:kworker.0 Request processed in 32us (494us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826417.649364:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14485 00000100:00100000:3.0:1787826417.649384:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346387072 00000100:00100000:3.0:1787826417.649389:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346387072 00000100:00100000:1.0:1787826417.649400:0:8752:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14486 00000100:00100000:1.0:1787826417.649410:0:8752:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346386944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.649422:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346387456 00000100:00100000:3.0:1787826417.649426:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346387456 00000100:00100000:3.0:1787826417.649451:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346387840 00000100:00100000:1.0:1787826417.649452:0:8752:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346386944:12345-0@lo:400:kworker.0 Request processed in 44us (693us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826417.649455:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346387840 00000100:00100000:1.0:1787826417.649463:0:8752:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14486 00000100:00100000:1.0:1787826417.649489:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8948 00000100:00100000:1.0:1787826417.649499:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346387072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.649504:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8949 00000100:00100000:3.0:1787826417.649515:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346387456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.649536:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346387072:12345-0@lo:400:kworker.0 Request processed in 40us (1017us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826417.649543:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8948 00000100:00100000:3.0:1787826417.649559:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346387456:12345-0@lo:400:kworker.0 Request processed in 46us (744us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826417.649562:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8950 00000100:00100000:3.0:1787826417.649571:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8949 00000100:00100000:1.0:1787826417.649576:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346387840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.649609:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346387840:12345-0@lo:400:kworker.0 Request processed in 38us (716us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826417.649615:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8950 00000100:00100000:1.0:1787826417.649640:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346386560 00000100:00100000:1.0:1787826417.649644:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346386560 00000001:00080000:3.0:1787826417.650377:0:9524:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294976177 is committed 00000100:00100000:3.0:1787826417.650425:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3652:1874663346387328:0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.650440:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346387840:0@lo:400:kworker.0 00000100:00100000:3.0:1787826417.650451:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3652:1874663346386816:0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.650596:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:lustre-MDT0000-lwp-MDT0001_UUID+5:3655:x1874663346386688:12345-0@lo:400:kworker.0 Request processed in 1496us (2574us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826417.650606:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14481 00000100:00100000:2.0:1787826417.650638:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346387584 00000100:00100000:2.0:1787826417.650643:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346387584 00000100:00100000:2.0:1787826417.650762:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24271 00000100:00100000:2.0:1787826417.650770:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346387584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.650823:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346387584:12345-0@lo:400:kworker.0 Request processed in 54us (2268us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826417.650829:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24271 00000100:00080000:2.0:1787826417.650862:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787826417.650867:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826417.650871:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826417.650873:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826417.651129:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24272 00000100:00100000:1.0:1787826417.651138:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346386560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.651452:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346386944:0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.651467:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346387456:0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.651479:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346387968:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.651543:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3655:x1874663346386560:12345-0@lo:400:kworker.0 Request processed in 406us (3066us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826417.651550:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24272 00000100:00100000:1.0:1787826417.651576:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346386432 00000100:00100000:1.0:1787826417.651580:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346386432 00000100:00100000:1.0:1787826417.651618:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2263 00000100:00100000:1.0:1787826417.651625:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977d60850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346386432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.651751:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977d60850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346386432:12345-0@lo:400:kworker.0 Request processed in 126us (3003us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826417.651758:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2263 00000100:00100000:1.0:1787826417.651819:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346386560:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.651831:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346387072:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.651842:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346387584:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.651855:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3655:1874663346386688:0@lo:400:kworker.0 00000100:00100000:1.0:1787826417.651869:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3655:1874663346387200:0@lo:400:kworker.0 00000100:00100000:2.0:1787826417.652062:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346386432:0@lo:400:kworker.0 00000100:00100000:0.0:1787826417.652617:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8951 00000100:00100000:0.0:1787826417.652628:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346387712:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826417.652689:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346387712:12345-0@lo:400:kworker.0 Request processed in 64us (4288us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826417.652696:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8951 00000100:00100000:3.0:1787826417.652713:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346387712:0@lo:400:kworker.0 00000100:00080000:2.0:1787826417.652745:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1787826417.844418:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826418.042174:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826418.042218:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332721024 00000100:00100000:0.0:1787826418.042227:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332721024 00000100:00100000:0.0:1787826418.042293:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68768 00000100:00100000:0.0:1787826418.042305:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:229244:x1874663332721024:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:0.0:1787826418.042332:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826418.042354:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc66 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d7f expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.042426:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cc6d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.042440:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cc6d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.042452:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cc6d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826418.042501:0:17813:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826418.042506:0:17813:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826418.042528:0:17813:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cc6d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.042554:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc66 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d7f expref: 14 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.042571:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cc6d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d7f expref: 13 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.042580:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826418.042584:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826418.042588:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43a00, rc 0) 00000100:00100000:0.0:1787826418.042785:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:229244:x1874663332721024:12345-192.168.203.56@tcp:101:link.0 Request processed in 481us (620us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826418.042793:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68768 00000100:00080000:0.0:1787826418.042817:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826418.053410:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826418.053630:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332721152 00000100:00100000:3.0:1787826418.053641:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332721152 00000100:00100000:3.0:1787826418.053708:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68769 00000100:00100000:3.0:1787826418.053719:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:229244:x1874663332721152:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:3.0:1787826418.053741:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826418.053763:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc74 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d86 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.053832:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.053846:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.053858:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.053871:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.053877:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.053884:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.053993:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc89 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054003:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054009:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787826418.054055:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6ea:0x0]: returning size 0 00000004:00200000:3.0:1787826418.054071:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826418.054092:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054136:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054148:0:6529:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054154:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054160:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054167:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054172:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826418.054178:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054184:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054196:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826418.054199:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826418.054203:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054209:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cc7b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054222:0:6529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054228:0:6529:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054234:0:6529:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054238:0:6529:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826418.054241:0:6529:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054247:0:6529:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054253:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x73b310ba (ffff9f784abfeb00) 00010000:00010000:3.0:1787826418.054256:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826418.054258:0:6529:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054276:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cc82 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054303:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68e00/0xe21309aa81f4cc74 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d86 expref: 15 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054316:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc89 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d86 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826418.054324:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:3.0:1787826418.054327:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787826418.054330:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e69400, rc 0) 00000100:00100000:3.0:1787826418.054388:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:229244:x1874663332721152:12345-192.168.203.56@tcp:101:link.0 Request processed in 669us (987us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826418.054395:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68769 00000100:00100000:3.0:1787826418.062960:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826418.064048:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332721408 00000100:00100000:0.0:1787826418.064072:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332721408 00000100:00100000:0.0:1787826418.064394:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68770 00000100:00100000:0.0:1787826418.064408:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:229244:x1874663332721408:12345-192.168.203.56@tcp:36:link.0 00010000:00010000:0.0:1787826418.064467:0:35080:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1787826418.064495:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69400/0xe21309aa81f4cc89 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d86 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.064514:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:0.0:1787826418.064524:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cc6d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d7f expref: 13 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.064534:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826418.064537:0:35080:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826418.064564:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.064577:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.064586:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.064712:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.064720:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826418.064727:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00020000:0.0:1787826418.064733:0:35080:0:(mdt_handler.c:4171:mdt_object_pdo_lock()) cfs_fail_timeout id 145 sleeping for 15000ms 00000100:00100000:0.0:1787826418.166717:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787826418.166744:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346388096:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826418.166822:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826418.168349:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346388096 00000100:00100000:1.0:1787826418.168361:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346388096 00000100:00100000:1.0:1787826418.168432:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6307 00000100:00100000:1.0:1787826418.168443:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346388096:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787826418.168478:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826418.168567:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346388096:12345-0@lo:13:osp-pre-1-1.0 Request processed in 125us (1745us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826418.168574:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6307 00000100:00100000:1.0:1787826418.168628:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346388096:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787826418.542882:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826418.543073:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332721536 00000100:00100000:2.0:1787826418.543084:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332721536 00000100:00100000:2.0:1787826418.543214:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68771 00000100:00100000:2.0:1787826418.543225:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:229246:x1874663332721536:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:2.0:1787826418.543246:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826418.543277:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4cc9e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d8d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826418.543352:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826418.543365:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 2/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826418.543372:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826418.543390:0:6528:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826418.543400:0:6528:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826418.543557:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826418.668698:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346388224:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826418.668782:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826418.668998:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346388352:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826418.669034:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826418.669067:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826418.669074:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346388480:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826418.669101:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826418.669155:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346388224 00000100:00100000:1.0:1787826418.669161:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346388224 00000100:00100000:1.0:1787826418.669215:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346388352 00000100:00100000:1.0:1787826418.669219:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346388352 00000100:00100000:2.0:1787826418.669225:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346388480 00000100:00100000:2.0:1787826418.669232:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346388480 00000100:00100000:1.0:1787826418.669240:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6308 00000100:00100000:1.0:1787826418.669252:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346388224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826418.669276:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:2.0:1787826418.669280:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24273 00000100:00100000:2.0:1787826418.669290:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346388480:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826418.669445:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:2.0:1787826418.669493:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346388480:12345-0@lo:41:osp-pre-0-1.0 Request processed in 204us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826418.669501:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24273 00000100:00100000:2.0:1787826418.669527:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346388608:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826418.669555:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826418.669619:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346388480:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826418.669661:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346388608 00000100:00100000:2.0:1787826418.669665:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346388608 00000100:00100000:2.0:1787826418.669690:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24274 00000100:00100000:2.0:1787826418.669697:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346388608:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787826418.669716:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:2.0:1787826418.669752:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346388608:12345-0@lo:41:osp-pre-1-0.0 Request processed in 55us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826418.669759:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24274 00000100:00100000:0.0:1787826418.671974:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7842c91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346388736:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826418.672050:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826418.674591:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346388736 00000100:00100000:2.0:1787826418.674619:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346388736 00000100:00100000:2.0:1787826418.674683:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6309 00000100:00100000:3.0:1787826418.674688:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346388608:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787826418.674699:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346388352:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1787826418.674738:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826418.674835:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346388352:12345-0@lo:13:osp-pre-0-1.0 Request processed in 140us (5800us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826418.674842:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6309 00000100:00100000:2.0:1787826418.674859:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6310 00000100:00100000:2.0:1787826418.674866:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346388736:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826418.674876:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826418.674913:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346388736:12345-0@lo:13:osp-pre-0-0.0 Request processed in 38us (2852us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826418.674919:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6310 00000100:00100000:3.0:1787826418.677046:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f79785e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346388352:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826418.677569:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346388224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 8316us (8783us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826418.677581:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6308 00000100:00100000:0.0:1787826418.678728:0:3655:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294976177 gen 2 00000004:00080000:0.0:1787826418.678737:0:3655:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f784a11ea00, transno 4294976177 00000100:00100000:0.0:1787826418.678771:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346388224:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826418.678805:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7842c91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346388736:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1787826418.860656:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826419.884652:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00020000:0.0:1787826420.215025:0:35080:0:(mdt_handler.c:4171:mdt_object_pdo_lock()) cfs_fail_timeout interrupted 00010000:00010000:0.0:1787826420.252006:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.252028:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.252042:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826420.255632:0:35080:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cef77ab0 reply state 00000000a7b72252 transno 8589957527 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255660:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255670:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:3.0:1787826420.255673:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00000100:00100000:3.0:1787826420.255677:0:35080:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787826420.255684:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255691:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255699:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255705:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255711:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826420.255716:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255723:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255733:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826420.255735:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826420.255739:0:35080:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255747:0:35080:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255753:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255782:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255789:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cc90 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826420.255802:0:35080:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cef77ab0 reply state 00000000a7b72252 transno 8589957527 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255810:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.255815:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4cd9242d (ffff9f784b4a5600) 00010000:00010000:3.0:1787826420.255817:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826420.255819:0:35080:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:3.0:1787826420.255945:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:229244:x1874663332721408:12345-192.168.203.56@tcp:36:link.0 Request processed in 2191537us (2192992us total) trans 8589957527 rc 0/0 00000100:00100000:3.0:1787826420.255954:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68770 00010000:00010000:0.0:1787826420.256645:0:6528:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.256656:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826420.256764:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826420.256769:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826420.256788:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.256819:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eea00/0xe21309aa81f4cc9e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.256839:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.256846:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826420.256849:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826420.256851:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ee600, rc 0) 00000100:00100000:0.0:1787826420.256909:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:229246:x1874663332721536:12345-192.168.203.56@tcp:101:link.0 Request processed in 1713685us (1714034us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826420.256916:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68771 00000100:00100000:1.0:1787826420.268774:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826420.271155:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332721664 00000100:00100000:0.0:1787826420.271171:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332721664 00000100:00080000:1.0:1787826420.271255:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787826420.271261:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68772 00000100:00100000:0.0:1787826420.271275:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:229246:x1874663332721664:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:0.0:1787826420.271295:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826420.271318:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ccb3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d94 expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271390:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271406:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271418:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271430:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271437:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271445:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271542:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271553:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271562:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826420.271610:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6ea:0x0]: returning size 0 00000004:00200000:0.0:1787826420.271616:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826420.271638:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271651:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271662:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271668:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271675:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271682:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271688:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826420.271693:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271700:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271714:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826420.271717:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826420.271722:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271733:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ccba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271745:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271751:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271757:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271762:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43e00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826420.271765:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271771:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271779:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4cd9242d (ffff9f784b4a5600) 00010000:00010000:0.0:1787826420.271781:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826420.271784:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271790:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4ccc1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271816:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ccb3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 9 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271834:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 8 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.271842:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:0.0:1787826420.271844:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826420.271847:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826420.271940:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:229246:x1874663332721664:12345-192.168.203.56@tcp:101:link.0 Request processed in 667us (3173us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826420.271947:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68772 00000100:00100000:2.0:1787826420.883895:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826420.884783:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332721792 00000100:00100000:1.0:1787826420.884795:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332721792 00000100:00100000:1.0:1787826420.884912:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68773 00000100:00100000:1.0:1787826420.884924:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:228930:x1874663332721792:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:1.0:1787826420.884951:0:24702:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:0.0:1787826420.884958:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787826420.884975:0:24702:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4cccf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801d9b expref: -99 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.885047:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ccd6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.885061:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ccd6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.885074:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ccd6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826420.885156:0:24702:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826420.885161:0:24702:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826420.885183:0:24702:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ccd6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.885210:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece000/0xe21309aa81f4cccf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d9b expref: 16 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.885229:0:24702:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ccd6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d9b expref: 15 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.885237:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826420.885240:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826420.885244:0:24702:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf800, rc 0) 00000100:00100000:1.0:1787826420.885321:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:228930:x1874663332721792:12345-192.168.203.56@tcp:101:bash.0 Request processed in 400us (1444us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826420.885328:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68773 00000100:00100000:1.0:1787826420.896732:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826420.897001:0:6531:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332721920 00000100:00100000:3.0:1787826420.897012:0:6531:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332721920 00000100:00100000:3.0:1787826420.897081:0:6531:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9413 00000100:00100000:3.0:1787826420.897092:0:6531:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:228930:x1874663332721920:12345-192.168.203.56@tcp:37:bash.0 00010000:00010000:3.0:1787826420.908605:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826420.910832:0:6531:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:228930:x1874663332721920:12345-192.168.203.56@tcp:37:bash.0 Request processed in 13731us (14098us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826420.910845:0:6531:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9413 00000100:00100000:2.0:1787826420.941616:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826420.942125:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332722048 00000100:00100000:1.0:1787826420.942138:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332722048 00000100:00100000:1.0:1787826420.942227:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68774 00000100:00100000:1.0:1787826420.942238:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:229287:x1874663332722048:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826420.942260:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826420.942282:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4ccdd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801da2 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942371:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942385:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942401:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942414:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942421:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942427:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942535:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ccf2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942545:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ccf2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942552:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ccf2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826420.942595:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6ea:0x0]: returning size 0 00000004:00200000:1.0:1787826420.942600:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826420.942622:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ccf2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942636:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ccf2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942647:0:35098:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942659:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942665:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942672:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942677:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826420.942682:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942690:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942702:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826420.942705:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826420.942709:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942716:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cce4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942731:0:35098:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942738:0:35098:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942744:0:35098:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942749:0:35098:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826420.942752:0:35098:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942758:0:35098:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942766:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4cd9242d (ffff9f784b4a5600) 00010000:00010000:1.0:1787826420.942768:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826420.942771:0:35098:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942777:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cceb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942808:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4ccdd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801da2 expref: 17 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942821:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ccf2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801da2 expref: 16 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.942828:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:1.0:1787826420.942831:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787826420.942833:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecfe00, rc 0) 00000100:00100000:1.0:1787826420.943311:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:229287:x1874663332722048:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1072us (1700us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826420.943319:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68774 00000100:00100000:1.0:1787826420.949458:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826420.949540:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332722176 00000100:00100000:3.0:1787826420.949551:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332722176 00000100:00100000:3.0:1787826420.949639:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30916 00000100:00100000:3.0:1787826420.949653:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332722176:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:3.0:1787826420.949670:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787826420.949692:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801da9 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787826420.949720:0:8445:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x525:0x0] initial LVB size: 0, mtime: 0x6a9010ef, atime: 0x6a9010ef, ctime: 0x6a9010ef, blocks: 0x0 00010000:00010000:3.0:1787826420.949743:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x525:0x0].0x0 since=8233s v=(0 0) 00010000:00010000:3.0:1787826420.949747:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x525:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826420.949757:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 6 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.949775:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 6 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.949799:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7846e68e00, rc 0) 00000100:00100000:3.0:1787826420.949866:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f79785e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332722176:12345-192.168.203.56@tcp:101:rm.0 Request processed in 216us (417us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826420.949873:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30916 00000100:00080000:0.0:1787826420.950002:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826420.964437:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826420.964550:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332722432 00000100:00100000:0.0:1787826420.964560:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332722432 00000100:00100000:0.0:1787826420.964627:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68775 00000100:00100000:0.0:1787826420.964638:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:229287:x1874663332722432:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826420.964690:0:17813:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826420.964718:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfe00/0xe21309aa81f4ccf2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801da2 expref: 16 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.964741:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecfc00/0xe21309aa81f4cc58 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6ea:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d78 expref: 15 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.964751:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:0.0:1787826420.964759:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4ccd6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d9b expref: 14 pid: 24702 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.964773:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826420.964776:0:17813:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826420.964799:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.964809:0:17813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.964816:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.964848:0:17813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.964860:0:17813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 7 pid: 6528 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826420.964876:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346388864:192.168.203.56@tcp:104: 00000100:00100000:0.0:1787826420.964988:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784c4f5e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826420.971822:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346388864:192.168.203.56@tcp:104: 00010000:00010000:0.0:1787826420.971855:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826420.973081:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826420.973210:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332722560 00000100:00100000:2.0:1787826420.973220:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332722560 00010000:00010000:2.0:1787826420.973265:0:6513:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 8 pid: 6528 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826420.973310:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22095 00000100:00100000:2.0:1787826420.973319:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2394:x1874663332722560:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826420.973335:0:6513:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826420.973342:0:6513:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 8 pid: 6528 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826420.973360:0:6513:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 8 pid: 6528 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826420.973383:0:6513:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee600/0xe21309aa81f4cca5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d8d expref: 8 pid: 6528 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826420.973402:0:6513:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826420.973407:0:6513:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826420.973418:0:6513:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826420.973425:0:6513:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826420.973458:0:6513:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787826420.973488:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.973496:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.973518:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826420.973525:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2394:x1874663332722560:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 208us (452us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826420.973528:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826420.973532:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22095 00010000:00010000:1.0:1787826420.973536:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.973645:0:17813:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.973657:0:17813:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.973665:0:17813:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 4/0,1 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.973698:0:17813:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 6 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.973711:0:17813:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 6 pid: 35094 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826420.973726:0:17813:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346388992:192.168.203.56@tcp:104: 00000100:00100000:1.0:1787826420.973781:0:17813:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f78500a0c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826420.976801:0:17813:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afca80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17813:1874663346388992:192.168.203.56@tcp:104: 00000100:00100000:2.0:1787826420.976801:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00010000:00010000:1.0:1787826420.976836:0:17813:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826420.977081:0:17812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332722688 00000100:00100000:3.0:1787826420.977088:0:17812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332722688 00010000:00010000:3.0:1787826420.977168:0:17812:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 7 pid: 35094 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826420.977198:0:17812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22096 00000100:00100000:3.0:1787826420.977207:0:17812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7971c83050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:2393:x1874663332722688:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826420.977221:0:17812:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826420.977228:0:17812:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 7 pid: 35094 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.977240:0:17812:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 7 pid: 35094 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.977264:0:17812:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ccc8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.56@tcp remote: 0x95194e8760801d94 expref: 7 pid: 35094 timeout: 8333 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.977279:0:17812:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:3.0:1787826420.977295:0:17812:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.977304:0:17812:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.977317:0:17812:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826420.977345:0:17812:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787826420.977401:0:17812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7971c83050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332722688:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 195us (606us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826420.977409:0:17812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22096 00010000:00010000:1.0:1787826420.977454:0:17813:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977462:0:17813:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977778:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977855:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977863:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977870:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826420.977875:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977883:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977903:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:1.0:1787826420.977906:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787826420.977910:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977920:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece800/0xe21309aa81f4cd0e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977937:0:17813:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977943:0:17813:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977948:0:17813:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977954:0:17813:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977958:0:17813:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826420.977961:0:17813:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977966:0:17813:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977974:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826420.977976:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826420.977979:0:17813:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.977985:0:17813:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cd00 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787826420.977995:0:17813:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003e82b426 reply state 00000000263bf493 transno 8589957528 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.978007:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826420.978013:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4cd9242d (ffff9f784b4a5600) 00010000:00010000:1.0:1787826420.978015:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787826420.978018:0:17813:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787826420.978066:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:229287:x1874663332722432:12345-192.168.203.56@tcp:36:rm.0 Request processed in 13429us (13637us total) trans 8589957528 rc 0/0 00000100:00100000:1.0:1787826420.978073:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68775 00000100:00100000:3.0:1787826420.982543:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826420.982640:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332722816 00000100:00100000:0.0:1787826420.982650:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332722816 00000100:00100000:0.0:1787826420.982719:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68776 00000100:00100000:0.0:1787826420.982730:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f540380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:229287:x1874663332722816:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826420.982753:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826420.982775:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4cd15 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801db0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.982842:0:6529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4cd1c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.982857:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4cd1c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.982868:0:6529:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4cd1c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826420.982923:0:6529:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826420.982927:0:6529:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826420.982947:0:6529:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4cd1c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6529 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.982973:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4cd15 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801db0 expref: 17 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.982991:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4cd1c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801db0 expref: 16 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.982999:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826420.983002:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826420.983005:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826420.983077:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f540380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:229287:x1874663332722816:12345-192.168.203.56@tcp:101:rm.0 Request processed in 348us (540us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826420.983084:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68776 00000100:00100000:2.0:1787826420.998034:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826420.998152:0:11629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332722944 00000100:00100000:0.0:1787826420.998163:0:11629:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332722944 00000100:00100000:0.0:1787826420.998233:0:11629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68777 00000100:00100000:0.0:1787826420.998244:0:11629:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7842c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:229287:x1874663332722944:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826420.998264:0:11629:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826420.998299:0:11629:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4cd23 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801db7 expref: -99 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998368:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998382:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998393:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998405:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998413:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998420:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998744:0:11629:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cd38 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998754:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cd38 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998766:0:11629:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cd38 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826420.998810:0:11629:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6ea:0x0]: returning size 0 00000004:00200000:0.0:1787826420.998816:0:11629:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826420.998836:0:11629:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cd38 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998849:0:11629:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cd38 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998860:0:11629:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998866:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998872:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998879:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998898:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826420.998902:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998910:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998922:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826420.998926:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826420.998931:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998936:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cd2a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998952:0:11629:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998958:0:11629:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998964:0:11629:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998969:0:11629:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826420.998972:0:11629:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998978:0:11629:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998986:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x73b310ba (ffff9f784abfeb00) 00010000:00010000:0.0:1787826420.998989:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826420.998992:0:11629:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.998997:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4cd31 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.999028:0:11629:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43e00/0xe21309aa81f4cd23 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801db7 expref: 18 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.999049:0:11629:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cd38 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801db7 expref: 17 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826420.999056:0:11629:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:0.0:1787826420.999059:0:11629:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826420.999062:0:11629:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43200, rc 0) 00000100:00100000:0.0:1787826420.999247:0:11629:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7842c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:398e0287-d3c7-44dd-b1d9-6b85f698e615+17:229287:x1874663332722944:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1003us (1222us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826420.999256:0:11629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68777 00000100:00100000:2.0:1787826421.013685:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826421.013781:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332723200 00000100:00100000:3.0:1787826421.013791:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332723200 00000100:00100000:3.0:1787826421.013860:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68778 00000100:00100000:3.0:1787826421.013870:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+17:229287:x1874663332723200:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:3.0:1787826421.013935:0:35080:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1787826421.013964:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cd38 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6ea:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801db7 expref: 17 pid: 11629 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.013990:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:3.0:1787826421.014000:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4cd1c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801db0 expref: 16 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014010:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826421.014014:0:35080:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826421.014040:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014053:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014066:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014077:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x73b310ba bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014085:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014092:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014383:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014395:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014402:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787826421.014649:0:35080:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].219, 64 off22208 00000004:00080000:3.0:1787826421.014660:0:35080:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 8233226553558ns, next at 8234226550993ns, rc = 0 00010000:00010000:3.0:1787826421.014709:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014719:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010080000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014726:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40010080000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014738:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40010080000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014745:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010480000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014750:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010480000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014883:0:35080:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826421.014917:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014924:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014930:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014936:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826421.014941:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014948:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014959:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:3.0:1787826421.014962:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826421.014966:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014976:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cd4d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6ea:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.014996:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015003:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015007:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015012:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015016:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826421.015019:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015025:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015032:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826421.015034:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826421.015037:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015043:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cd3f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826421.015052:0:35080:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000093b5d47e reply state 00000000ed5c833a transno 8589957529 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015064:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826421.015070:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x73b310ba (ffff9f784abfeb00) 00010000:00010000:3.0:1787826421.015073:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826421.015076:0:35080:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787826421.015508:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+17:229287:x1874663332723200:12345-192.168.203.56@tcp:36:rm.0 Request processed in 1634us (1829us total) trans 8589957529 rc 0/0 00000100:00100000:3.0:1787826421.015517:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68778 00010000:00010000:2.0:1787826421.018359:0:6515:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826421.018376:0:6515:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826421.018388:0:6515:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68400 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:2.0:1787826421.018392:0:6515:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826421.018401:0:6515:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826421.018420:0:6515:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:2.0:1787826421.018422:0:6515:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1787826421.018427:0:6515:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826421.018432:0:6515:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68400/0xe21309aa81f4cd54 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6ea:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826421.018450:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826421.018452:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826421.018453:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826421.932826:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826421.989416:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826421.989483:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826421.989538:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826421.989568:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826421.989589:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826421.989622:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826421.989649:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826421.989687:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332723584 00000100:00100000:2.0:1787826421.989699:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332723584 00000100:00100000:2.0:1787826421.989776:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332724096 00000100:00100000:2.0:1787826421.989780:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332724096 00000100:00100000:2.0:1787826421.989800:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332723456 00000100:00100000:2.0:1787826421.989803:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332723456 00000100:00100000:2.0:1787826421.989823:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332723968 00000100:00100000:2.0:1787826421.989826:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332723968 00000100:00100000:2.0:1787826421.989848:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14487 00000100:00100000:2.0:1787826421.989860:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332723584:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826421.989956:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784a11ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332723584:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 99us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826421.989964:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14487 00000100:00100000:2.0:1787826421.989983:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14488 00000100:00100000:2.0:1787826421.989989:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332724096:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826421.990011:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332724096:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 23us (443us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826421.990018:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14488 00000100:00100000:2.0:1787826421.990029:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14489 00000100:00100000:2.0:1787826421.990035:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+17:2393:x1874663332723456:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826421.990066:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+17:2393:x1874663332723456:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 30us (475us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826421.990072:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14489 00000100:00100000:2.0:1787826421.990083:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14490 00000100:00100000:2.0:1787826421.990089:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332723968:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826421.990157:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332723968:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 67us (506us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826421.990163:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14490 00000100:00100000:0.0:1787826421.990364:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332723328 00000100:00100000:0.0:1787826421.990369:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332723328 00000100:00100000:0.0:1787826421.990410:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2264 00000100:00100000:0.0:1787826421.990418:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847253450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2393:x1874663332723328:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826421.990454:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847253450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2393:x1874663332723328:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 37us (1050us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826421.990461:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2264 00000100:00080000:0.0:1787826421.990499:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826421.990952:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332723840 00000100:00100000:3.0:1787826421.990959:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332723840 00000100:00100000:3.0:1787826421.991053:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332723712 00000100:00100000:3.0:1787826421.991056:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332723712 00000100:00100000:3.0:1787826421.991083:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8952 00000100:00100000:3.0:1787826421.991090:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332723840:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826421.991193:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332723840:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 103us (1655us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826421.991200:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8952 00000100:00100000:3.0:1787826421.991216:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8953 00000100:00100000:3.0:1787826421.991223:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332723712:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826421.991248:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332723712:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 25us (1625us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826421.991253:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8953 00000100:00080000:0.0:1787826421.991408:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787826422.766574:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346389504:0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.766584:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346389120:0@lo:400:kworker.0 00000100:00100000:2.0:1787826422.767839:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346389248:0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.768092:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826422.768150:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346389632:0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.768179:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826422.768203:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346390144:0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.768218:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826422.768239:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346390016:0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.768240:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826422.768258:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346390656:0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.768262:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826422.768279:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826422.768289:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346390528:0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.768309:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784f541880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346389376:0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.768310:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826422.768333:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826422.768341:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346389632 00000100:00100000:0.0:1787826422.768348:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346389632 00000100:00100000:3.0:1787826422.768350:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346389888:0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.768369:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826422.768401:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346389504 00000100:00100000:0.0:1787826422.768404:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346389504 00000100:00100000:0.0:1787826422.768424:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346390016 00000100:00100000:0.0:1787826422.768426:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346390016 00000100:00100000:0.0:1787826422.768445:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346390656 00000100:00100000:0.0:1787826422.768448:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346390656 00000100:00100000:0.0:1787826422.768462:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346389376 00000100:00100000:0.0:1787826422.768465:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346389376 00000100:00100000:0.0:1787826422.768480:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346389888 00000100:00100000:0.0:1787826422.768483:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346389888 00000100:00100000:0.0:1787826422.768505:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14491 00000100:00100000:0.0:1787826422.768514:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346389632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.768691:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346390400:0@lo:400:kworker.0 00000100:00100000:2.0:1787826422.768944:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826422.768977:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346389760:0@lo:400:kworker.0 00000100:00100000:2.0:1787826422.769000:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826422.769012:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874663346389632:12345-0@lo:400:kworker.0 Request processed in 499us (831us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826422.769017:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346390272:0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769021:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14491 00000100:00100000:0.0:1787826422.769043:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14492 00000100:00100000:2.0:1787826422.769049:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826422.769050:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346389504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769083:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3655:x1874663346389504:12345-0@lo:400:kworker.0 Request processed in 35us (865us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826422.769088:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14492 00000100:00100000:0.0:1787826422.769101:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14493 00000100:00100000:0.0:1787826422.769125:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346390016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769215:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3655:x1874663346390016:12345-0@lo:400:kworker.0 Request processed in 91us (952us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826422.769222:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14493 00000100:00100000:0.0:1787826422.769235:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14494 00000100:00100000:0.0:1787826422.769241:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346390656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769268:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3652:x1874663346390656:12345-0@lo:400:kworker.0 Request processed in 27us (989us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826422.769274:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14494 00000100:00100000:0.0:1787826422.769284:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14495 00000100:00100000:0.0:1787826422.769289:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346389376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769312:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346389376:12345-0@lo:400:kworker.0 Request processed in 23us (978us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826422.769316:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14495 00000100:00100000:0.0:1787826422.769326:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14496 00000100:00100000:0.0:1787826422.769331:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346389888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769357:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346389888:12345-0@lo:400:kworker.0 Request processed in 27us (988us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826422.769363:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14496 00000100:00100000:0.0:1787826422.769391:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346390144 00000100:00100000:0.0:1787826422.769396:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346390144 00000100:00100000:0.0:1787826422.769426:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346390528 00000100:00100000:0.0:1787826422.769429:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346390528 00000100:00100000:0.0:1787826422.769449:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346389760 00000100:00100000:0.0:1787826422.769452:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346389760 00000100:00100000:0.0:1787826422.769470:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8954 00000100:00100000:0.0:1787826422.769476:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346390144:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769507:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346390144:12345-0@lo:400:kworker.0 Request processed in 31us (1266us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826422.769512:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8954 00000100:00100000:0.0:1787826422.769524:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8955 00000100:00100000:0.0:1787826422.769530:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346390528:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769559:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346390528:12345-0@lo:400:kworker.0 Request processed in 30us (1247us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826422.769565:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8955 00000100:00100000:0.0:1787826422.769576:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8956 00000100:00100000:0.0:1787826422.769582:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346389760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769611:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346389760:12345-0@lo:400:kworker.0 Request processed in 29us (610us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826422.769616:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8956 00000100:00080000:0.0:1787826422.769636:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787826422.769640:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787826422.769643:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787826422.769645:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826422.769704:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346390528:0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769717:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3655:1874663346389504:0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.769727:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3655:1874663346390016:0@lo:400:kworker.0 00000100:00100000:1.0:1787826422.770607:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346389248 00000100:00100000:1.0:1787826422.770616:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346389248 00000100:00100000:1.0:1787826422.770654:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346390272 00000100:00100000:1.0:1787826422.770656:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346390272 00000100:00100000:1.0:1787826422.770675:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24275 00000100:00100000:1.0:1787826422.770683:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346389248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826422.770736:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346389248:12345-0@lo:400:kworker.0 Request processed in 55us (1793us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826422.770742:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24275 00000100:00100000:1.0:1787826422.770755:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24276 00000100:00100000:1.0:1787826422.770762:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346390272:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826422.770788:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346390272:12345-0@lo:400:kworker.0 Request processed in 27us (1738us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826422.770793:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24276 00000001:00080000:0.0:1787826422.772331:0:30894:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589957529 is committed 00000004:00080000:0.0:1787826422.772345:0:30894:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 218 committed 00000100:00100000:0.0:1787826422.772429:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3652:1874663346389632:0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.772444:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346390144:0@lo:400:kworker.0 00000100:00100000:0.0:1787826422.772456:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3652:1874663346390656:0@lo:400:kworker.0 00010000:00010000:1.0:1787826422.772516:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.772528:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.772537:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.772543:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490efe00 already unused, calling callback (ffffffffc197a4f0) 00000100:00100000:2.0:1787826422.773188:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f543100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346389248:0@lo:400:kworker.0 00000100:00100000:2.0:1787826422.773203:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346389760:0@lo:400:kworker.0 00000100:00100000:2.0:1787826422.773215:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346390272:0@lo:400:kworker.0 00010000:00010000:1.0:1787826422.773495:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773505:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773524:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x73b310ba (ffff9f784abfeb00) 00010000:00010000:1.0:1787826422.773527:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826422.773532:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773538:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490efe00/0xe21309aa81f4cc3c lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11629 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773569:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773576:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773582:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773588:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826422.773590:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773600:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773607:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:1.0:1787826422.773610:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826422.773613:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773618:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4cc51 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773629:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773635:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773641:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773646:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826422.773648:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773654:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773661:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4cd9242d (ffff9f784b4a5600) 00010000:00010000:1.0:1787826422.773663:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826422.773666:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773673:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43400/0xe21309aa81f4cc97 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773681:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773686:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773693:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773697:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42400 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826422.773700:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773706:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773713:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6ea:0x0].0x0 (ffff9f784b4a5800) 00010000:00010000:1.0:1787826422.773715:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00010000:00010000:1.0:1787826422.773718:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773722:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4ccac lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x6ea:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773740:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773745:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773751:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773756:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7848ece200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826422.773760:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773766:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773774:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4cd9242d (ffff9f784b4a5600) 00010000:00010000:1.0:1787826422.773776:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826422.773779:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773784:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece200/0xe21309aa81f4cd07 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x4cd9242d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 17813 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773796:0:3639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773802:0:3639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773807:0:3639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773813:0:3639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:1.0:1787826422.773815:0:3639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.773825:0:3639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.774314:0:3639:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x73b310ba (ffff9f784abfeb00) 00010000:00010000:1.0:1787826422.774317:0:3639:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787826422.774321:0:3639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826422.774328:0:3639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69c00/0xe21309aa81f4cd46 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x73b310ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826422.774516:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346389120 00000100:00100000:3.0:1787826422.774524:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346389120 00000100:00100000:3.0:1787826422.774639:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2265 00000100:00100000:3.0:1787826422.774649:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847253850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346389120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.774735:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847253850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3652:x1874663346389120:12345-0@lo:400:kworker.0 Request processed in 88us (6642us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826422.774743:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2265 00000100:00100000:3.0:1787826422.774894:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f542d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4305192b-43a6-4bf4-9f25-68faf720ed51:3652:1874663346389120:0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.774916:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826422.774926:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851243100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346390784:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1787826422.774961:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826422.774995:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346390784 00000100:00100000:3.0:1787826422.774998:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346390784 00000100:00100000:3.0:1787826422.775032:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30917 00000100:00100000:3.0:1787826422.775039:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346390784:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1787826422.775059:0:20167:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:1317 count 1 00010000:00010000:3.0:1787826422.775081:0:20167:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.775102:0:20167:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 5 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.775135:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x525:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1787826422.775139:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x525:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826422.775144:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 4/0,1 mode: --/PW res: [0x280000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.775170:0:20167:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 5 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.775184:0:20167:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 5 pid: 8445 timeout: 8334 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826422.775196:0:20167:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851241180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346390912:192.168.203.56@tcp:104: 00000100:00080000:2.0:1787826422.775836:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826422.776463:0:20167:0:(client.c:2715:ptlrpc_set_wait()) set ffff9f784f737e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826422.778492:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826422.778589:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784f541880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346389376:0@lo:400:kworker.0 00000100:00100000:2.0:1787826422.778604:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346389888:0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.778753:0:24218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346390400 00000100:00100000:3.0:1787826422.778760:0:24218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346390400 00000100:00100000:3.0:1787826422.778817:0:24218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8957 00000100:00100000:3.0:1787826422.778828:0:24218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346390400:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826422.778907:0:24218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346390400:12345-0@lo:400:kworker.0 Request processed in 80us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826422.778915:0:24218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8957 00000100:00100000:2.0:1787826422.778983:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346390400:0@lo:400:kworker.0 00000100:00100000:2.0:1787826422.780233:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826422.780375:0:6511:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332724224 00000100:00100000:3.0:1787826422.780383:0:6511:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332724224 00010000:00010000:3.0:1787826422.780439:0:6511:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 6 pid: 8445 timeout: 8334 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826422.780483:0:6511:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 22097 00000100:00100000:3.0:1787826422.780493:0:6511:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332724224:12345-192.168.203.56@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826422.780506:0:6511:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787826422.780513:0:6511:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x525:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 6 pid: 8445 timeout: 8334 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.780527:0:6511:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x525:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 6 pid: 8445 timeout: 8334 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.780553:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x525:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826422.780558:0:6511:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x525:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787826422.780564:0:6511:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.780575:0:6511:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.780595:0:6511:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00080000:2.0:1787826422.780627:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787826422.780635:0:6511:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332724224:12345-192.168.203.56@tcp:103:ldlm_bl.0 Request processed in 143us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826422.780646:0:6511:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 22097 00010000:00010000:2.0:1787826422.781903:0:20167:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000820f688e/0xe21309aa81f4ccf9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x525:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.56@tcp remote: 0x95194e8760801da9 expref: 5 pid: 8445 timeout: 8334 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826422.781925:0:20167:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851241180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20167:1874663346390912:192.168.203.56@tcp:104: 00010000:00010000:2.0:1787826422.781942:0:20167:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826422.781952:0:20167:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826422.781959:0:20167:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826422.781969:0:20167:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826422.781977:0:20167:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826422.782003:0:20167:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826422.782052:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826422.782059:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.782068:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69600 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826422.782073:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.782082:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.782100:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.782124:0:7478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027ed3819/0xe21309aa81f4cd5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x525:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826422.782156:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826422.782158:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826422.782160:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826422.782945:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346390784:12345-0@lo:6:osp-syn-0-0.0 Request processed in 7902us (7979us total) trans 4294973870 rc 0/0 00000100:00100000:2.0:1787826422.782957:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30917 00000004:00080000:3.0:1787826422.783010:0:3652:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9f7851243100/2, rc 0, transno 4294973870 00000100:00100000:3.0:1787826422.783021:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851243100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346390784:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1787826422.956391:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826423.280945:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826423.280971:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346391040:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826423.281046:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826423.281240:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346391040 00000100:00100000:3.0:1787826423.281249:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346391040 00000100:00100000:3.0:1787826423.281322:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6311 00000100:00100000:3.0:1787826423.281345:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346391040:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826423.281375:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826423.281428:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346391040:12345-0@lo:13:osp-pre-1-1.0 Request processed in 97us (382us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826423.281434:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6311 00000100:00100000:1.0:1787826423.281823:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346391040:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826423.788252:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787826423.788278:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346391168:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787826423.788428:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826423.788571:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346391168 00000100:00100000:1.0:1787826423.788580:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346391168 00000100:00100000:1.0:1787826423.788639:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6312 00000100:00100000:1.0:1787826423.788649:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346391168:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826423.788673:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826423.788810:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346391296:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787826423.788843:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826423.788864:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826423.788881:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346391424:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787826423.788906:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826423.788931:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346391424 00000100:00100000:0.0:1787826423.788935:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346391424 00000100:00100000:0.0:1787826423.788955:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6313 00000100:00100000:0.0:1787826423.788960:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346391424:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787826423.788979:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826423.789410:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346391168:12345-0@lo:13:osp-pre-1-0.0 Request processed in 761us (981us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826423.789418:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6312 00000100:00100000:1.0:1787826423.789453:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346391296 00000100:00100000:1.0:1787826423.789458:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346391296 00000100:00100000:1.0:1787826423.789487:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24277 00000100:00100000:1.0:1787826423.789494:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346391296:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787826423.789520:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:1.0:1787826423.789556:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346391296:12345-0@lo:41:osp-pre-0-1.0 Request processed in 63us (712us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826423.789562:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24277 00000100:00100000:1.0:1787826423.789582:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346391680:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787826423.789608:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826423.789635:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826423.789641:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851242300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346391552:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826423.789663:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826423.789688:0:8152:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346391552 00000100:00100000:1.0:1787826423.789692:0:8152:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346391552 00000100:00100000:0.0:1787826423.789812:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346391424:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (118us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826423.789822:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6313 00000100:00100000:0.0:1787826423.789838:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346391680 00000100:00100000:0.0:1787826423.789841:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346391680 00000100:00100000:0.0:1787826423.789863:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6314 00000100:00100000:0.0:1787826423.789879:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346391680:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787826423.789892:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826423.789932:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346391680:12345-0@lo:13:osp-pre-0-1.0 Request processed in 54us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826423.789938:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6314 00000100:00100000:0.0:1787826423.789983:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346391424:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826423.789994:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346391680:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787826423.789997:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346391296:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826423.790696:0:8152:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24278 00000100:00100000:1.0:1787826423.790711:0:8152:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346391552:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826423.790827:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346391168:0@lo:13:osp-pre-1-0.0 00000004:00000024:1.0:1787826423.790920:0:8152:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:1.0:1787826423.790981:0:8152:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346391552:12345-0@lo:41:osp-pre-1-0.0 Request processed in 274us (1316us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826423.790988:0:8152:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24278 00000100:00100000:0.0:1787826423.791434:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851242300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346391552:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1787826423.983915:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826425.004987:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826426.032978:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826427.052428:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826427.105425:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826427.105529:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826427.105563:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826427.105585:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826427.105602:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826427.105620:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826427.105643:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826427.105674:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332724864 00000100:00100000:1.0:1787826427.105685:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332724864 00000100:00100000:1.0:1787826427.105752:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332724736 00000100:00100000:1.0:1787826427.105755:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332724736 00000100:00100000:1.0:1787826427.105781:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8958 00000100:00100000:1.0:1787826427.105794:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845affb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332724864:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826427.105896:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845affb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332724864:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 105us (332us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826427.105904:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8958 00000100:00100000:1.0:1787826427.105925:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8959 00000100:00100000:1.0:1787826427.105932:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332724736:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826427.105951:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332724352 00000100:00100000:2.0:1787826427.105956:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332724352 00000100:00100000:1.0:1787826427.105962:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332724736:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 31us (341us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826427.105967:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8959 00000100:00100000:1.0:1787826427.105989:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332724608 00000100:00100000:1.0:1787826427.105992:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332724608 00000100:00100000:2.0:1787826427.105996:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2266 00000100:00100000:2.0:1787826427.106003:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847251050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2393:x1874663332724352:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826427.106023:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332725120 00000100:00100000:1.0:1787826427.106026:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332725120 00000100:00100000:2.0:1787826427.106030:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847251050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2393:x1874663332724352:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 28us (620us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826427.106037:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2266 00000100:00100000:1.0:1787826427.106043:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332724480 00000100:00100000:1.0:1787826427.106046:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332724480 00000100:00100000:1.0:1787826427.106062:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332724992 00000100:00100000:1.0:1787826427.106065:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332724992 00000100:00080000:2.0:1787826427.106065:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826427.106069:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787826427.106070:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787826427.106073:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1787826427.106075:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826427.106086:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14497 00000100:00100000:1.0:1787826427.106093:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332724608:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826427.106337:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14498 00000100:00100000:3.0:1787826427.106347:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332725120:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826427.106377:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2393:x1874663332725120:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 33us (791us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826427.106384:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14498 00000100:00100000:3.0:1787826427.106400:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14499 00000100:00100000:3.0:1787826427.106406:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332724480:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826427.106430:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845aff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332724480:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (827us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826427.106436:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14499 00000100:00100000:3.0:1787826427.106448:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14500 00000100:00100000:3.0:1787826427.106454:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332724992:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826427.106478:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2393:x1874663332724992:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 24us (834us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826427.106484:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14500 00000100:00100000:1.0:1787826427.106565:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7845afc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332724608:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 472us (1036us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826427.106571:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14497 00000001:02000400:1.0:1787826427.427605:0:169127:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 46d: pdirops: link vs unlink ======================================================================== 06:27:05 (1787826425) 00000100:00100000:1.0:1787826427.886005:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346391936:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886101:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826427.886194:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346392448:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886226:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826427.886252:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346392960:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886274:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826427.886300:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:1.0:1787826427.886306:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346391808:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886333:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826427.886353:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346392320:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886391:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826427.886409:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346392832:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886431:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826427.886447:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346393344:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886467:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826427.886500:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346391936 00000100:00100000:1.0:1787826427.886510:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346391936 00000100:00100000:1.0:1787826427.886556:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346392960 00000100:00100000:1.0:1787826427.886559:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346392960 00000100:00100000:1.0:1787826427.886581:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24279 00000100:00100000:1.0:1787826427.886590:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346391936:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886649:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3653:x1874663346391936:12345-0@lo:400:kworker.0 Request processed in 59us (548us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826427.886657:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24279 00000100:00100000:1.0:1787826427.886674:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24280 00000100:00100000:1.0:1787826427.886681:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346392960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886709:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346392960:12345-0@lo:400:kworker.0 Request processed in 29us (434us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826427.886714:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24280 00000100:00100000:1.0:1787826427.886736:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346392448 00000100:00100000:1.0:1787826427.886740:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346392448 00000100:00100000:1.0:1787826427.886779:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346392832 00000100:00100000:1.0:1787826427.886783:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346392832 00000100:00100000:1.0:1787826427.886805:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8960 00000100:00100000:1.0:1787826427.886811:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346392448:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886858:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346392448:12345-0@lo:400:kworker.0 Request processed in 47us (630us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826427.886866:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8960 00000100:00100000:1.0:1787826427.886880:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8961 00000100:00100000:1.0:1787826427.886887:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346392832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.886918:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346392832:12345-0@lo:400:kworker.0 Request processed in 32us (486us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826427.886924:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8961 00000100:00080000:1.0:1787826427.886949:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787826427.886954:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787826427.886993:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346392832:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.887006:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346391936:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.887016:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346392448:0@lo:400:kworker.0 00000100:00100000:1.0:1787826427.887024:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346392960:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887165:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346392064:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887203:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826427.887221:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346392576:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887241:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826427.887259:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346393088:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887280:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826427.887304:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1787826427.887310:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346392192:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887339:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826427.887356:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346392704:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887384:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826427.887400:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346393216:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887428:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826427.887463:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346392320 00000100:00100000:0.0:1787826427.887468:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346392320 00000100:00100000:0.0:1787826427.887512:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346393344 00000100:00100000:0.0:1787826427.887516:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346393344 00000100:00100000:0.0:1787826427.887537:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346392064 00000100:00100000:0.0:1787826427.887540:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346392064 00000100:00100000:0.0:1787826427.887558:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346392576 00000100:00100000:0.0:1787826427.887561:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346392576 00000100:00100000:0.0:1787826427.887578:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346392192 00000100:00100000:0.0:1787826427.887580:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346392192 00000100:00100000:0.0:1787826427.887599:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346392704 00000100:00100000:0.0:1787826427.887602:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346392704 00000100:00100000:0.0:1787826427.887625:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14501 00000100:00100000:0.0:1787826427.887632:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b753100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346392320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887672:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b753100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874663346392320:12345-0@lo:400:kworker.0 Request processed in 41us (1281us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826427.887682:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14501 00000100:00100000:0.0:1787826427.887696:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14502 00000100:00100000:0.0:1787826427.887702:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b752680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346393344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887730:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b752680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874663346393344:12345-0@lo:400:kworker.0 Request processed in 28us (1261us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826427.887737:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14502 00000100:00100000:0.0:1787826427.887747:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14503 00000100:00100000:0.0:1787826427.887753:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346392064:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887782:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0001_UUID+5:3654:x1874663346392064:12345-0@lo:400:kworker.0 Request processed in 29us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826427.887789:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14503 00000100:00100000:0.0:1787826427.887800:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14504 00000100:00100000:0.0:1787826427.887806:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346392576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887831:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f541500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0001_UUID+5:3654:x1874663346392576:12345-0@lo:400:kworker.0 Request processed in 26us (589us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826427.887849:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14504 00000100:00100000:0.0:1787826427.887861:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14505 00000100:00100000:0.0:1787826427.887867:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346392192:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887894:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346392192:12345-0@lo:400:kworker.0 Request processed in 27us (554us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826427.887899:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14505 00000100:00100000:0.0:1787826427.887911:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14506 00000100:00100000:0.0:1787826427.887917:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346392704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.887942:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346392704:12345-0@lo:400:kworker.0 Request processed in 26us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826427.887947:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14506 00000100:00100000:0.0:1787826427.887982:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346393088 00000100:00100000:0.0:1787826427.887987:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346393088 00000100:00100000:0.0:1787826427.888012:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346393216 00000100:00100000:0.0:1787826427.888014:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346393216 00000100:00100000:0.0:1787826427.888030:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8962 00000100:00100000:0.0:1787826427.888035:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346393088:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888066:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784f542a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+6:3654:x1874663346393088:12345-0@lo:400:kworker.0 Request processed in 30us (784us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826427.888072:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8962 00000100:00100000:0.0:1787826427.888082:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8963 00000100:00100000:0.0:1787826427.888089:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346393216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888140:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346393216:12345-0@lo:400:kworker.0 Request processed in 51us (710us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826427.888145:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8963 00000100:00080000:0.0:1787826427.888187:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787826427.888190:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787826427.888193:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787826427.888196:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787826427.888223:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346392192:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888234:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346392704:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888244:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346393216:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888253:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3654:1874663346392064:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888262:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3654:1874663346392576:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888274:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346393088:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888320:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3653:1874663346392320:0@lo:400:kworker.0 00000100:00100000:0.0:1787826427.888331:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874663346393344:0@lo:400:kworker.0 00000100:00100000:3.0:1787826427.888944:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346391808 00000100:00100000:3.0:1787826427.888954:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346391808 00000100:00100000:3.0:1787826427.889004:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2267 00000100:00100000:3.0:1787826427.889013:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847253050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346391808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826427.889076:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847253050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3653:x1874663346391808:12345-0@lo:400:kworker.0 Request processed in 65us (2741us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826427.889083:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2267 00000100:00100000:3.0:1787826427.889183:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4305192b-43a6-4bf4-9f25-68faf720ed51:3653:1874663346391808:0@lo:400:kworker.0 00010000:00010000:0.0:1787826428.079464:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826428.396546:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346393472:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787826428.396635:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826428.396708:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346393472 00000100:00100000:0.0:1787826428.396718:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346393472 00000100:00100000:0.0:1787826428.396786:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6315 00000100:00100000:0.0:1787826428.396798:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346393472:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787826428.396843:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:0.0:1787826428.396896:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346393472:12345-0@lo:13:osp-pre-1-1.0 Request processed in 99us (261us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826428.396904:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6315 00000100:00100000:0.0:1787826428.396962:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784836c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346393472:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1787826428.611644:0:169175:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0000_UUID 00010000:00010000:3.0:1787826428.611721:0:169175:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace mdt-lustre-MDT0001_UUID 00000100:00100000:3.0:1787826428.909189:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787826428.909216:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346393600:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826428.909297:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826428.909395:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346393600 00000100:00100000:1.0:1787826428.909406:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346393600 00000100:00100000:1.0:1787826428.909468:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6316 00000100:00100000:1.0:1787826428.909479:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346393600:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1787826428.909512:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:1.0:1787826428.909564:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346393600:12345-0@lo:13:osp-pre-0-1.0 Request processed in 87us (267us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826428.909572:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6316 00000100:00100000:1.0:1787826428.909620:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346393600:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826428.913523:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346393856:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826428.913609:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826428.913691:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346393856 00000100:00100000:2.0:1787826428.913700:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346393856 00000100:00100000:2.0:1787826428.913754:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6317 00000100:00100000:2.0:1787826428.913766:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346393856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787826428.913792:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826428.913850:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3652:x1874663346393856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826428.913861:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6317 00000100:00100000:2.0:1787826428.913887:0:3654:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787826428.913896:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7847b58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346393728:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826428.913928:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826428.913961:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346393728 00000100:00100000:2.0:1787826428.913965:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346393728 00000100:00100000:2.0:1787826428.913993:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24281 00000100:00100000:2.0:1787826428.914000:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346393728:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787826428.914029:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:2.0:1787826428.914065:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3654:x1874663346393728:12345-0@lo:41:osp-pre-0-1.0 Request processed in 65us (136us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826428.914071:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24281 00000100:00100000:2.0:1787826428.914135:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3654:1874663346393728:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826428.914171:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7847b5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346393856:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826428.914483:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346393984:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787826428.914520:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826428.914877:0:6533:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346393984 00000100:00100000:1.0:1787826428.914883:0:6533:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346393984 00000100:00100000:1.0:1787826428.914920:0:6533:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24282 00000100:00100000:1.0:1787826428.914928:0:6533:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346393984:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1787826428.914950:0:6533:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:1.0:1787826428.914993:0:6533:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851242300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346393984:12345-0@lo:41:osp-pre-1-0.0 Request processed in 66us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826428.915000:0:6533:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24282 00000100:00100000:1.0:1787826428.915042:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7851240000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346393984:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787826428.915058:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826428.915067:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346394112:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826428.915096:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826428.915151:0:8446:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346394112 00000100:00100000:1.0:1787826428.915155:0:8446:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346394112 00000100:00100000:1.0:1787826428.915180:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6318 00000100:00100000:1.0:1787826428.915186:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346394112:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1787826428.915202:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8645184 pending 0 free 3914715136 avail 3683192832 00000100:00100000:1.0:1787826428.915233:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346394112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 47us (136us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826428.915240:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6318 00000100:00100000:1.0:1787826428.915267:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346394112:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1787826428.926960:0:8438:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973870 is committed 00010000:00010000:1.0:1787826429.104188:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826429.746905:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826429.747036:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332725248 00000100:00100000:1.0:1787826429.747050:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332725248 00000100:00100000:1.0:1787826429.747160:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68779 00000100:00100000:1.0:1787826429.747175:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:229971:x1874663332725248:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:1.0:1787826429.747202:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826429.747226:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cd62 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801dbe expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826429.747300:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4cd69 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826429.747316:0:35095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4cd69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826429.747328:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4cd69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826429.747379:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826429.747383:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826429.747405:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4cd69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826429.747432:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece400/0xe21309aa81f4cd62 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dbe expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826429.747450:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4cd69 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dbe expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826429.747461:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826429.747464:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826429.747469:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecf400, rc 0) 00000100:00100000:1.0:1787826429.747533:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:229971:x1874663332725248:12345-192.168.203.56@tcp:101:touch.0 Request processed in 362us (644us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826429.747540:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68779 00000100:00080000:0.0:1787826429.748244:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826429.750429:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826429.750511:0:24703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332725504 00000100:00100000:0.0:1787826429.750523:0:24703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332725504 00000100:00100000:0.0:1787826429.750591:0:24703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68780 00000100:00100000:0.0:1787826429.750602:0:24703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:229971:x1874663332725504:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:0.0:1787826429.750726:0:24703:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826429.750731:0:24703:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787826429.750764:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf400/0xe21309aa81f4cd69 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dbe expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.750785:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826429.750790:0:24703:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826429.750806:0:24703:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4cd70 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801dc5 expref: -99 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.750954:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.750969:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.750979:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.750995:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cd7e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.751002:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cd7e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.751009:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cd7e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787826429.752051:0:24703:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x526:0x0], index 1: 0 00010000:00010000:0.0:1787826429.754390:0:24703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4cd85 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754420:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4cd85 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754441:0:24703:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4cd85 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754484:0:24703:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754493:0:24703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754501:0:24703:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754508:0:24703:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754515:0:24703:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826429.754520:0:24703:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754528:0:24703:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754544:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826429.754547:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826429.754552:0:24703:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754559:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cd77 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826429.754574:0:24703:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f91622be reply state 00000000804734f1 transno 8589957530 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cd7e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754587:0:24703:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42c00/0xe21309aa81f4cd7e lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754592:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x14efb6f5 (ffff9f794b57a000) 00010000:00010000:0.0:1787826429.754595:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826429.754598:0:24703:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787826429.754840:0:24703:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42800/0xe21309aa81f4cd70 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dc5 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754858:0:24703:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4cd85 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dc5 expref: 8 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826429.754867:0:24703:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:0.0:1787826429.754870:0:24703:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826429.754873:0:24703:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43000, rc 0) 00000100:00100000:1.0:1787826429.758213:0:24703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851241180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:229971:x1874663332725504:12345-192.168.203.56@tcp:101:touch.0 Request processed in 7598us (7784us total) trans 8589957530 rc 0/0 00000100:00100000:1.0:1787826429.758229:0:24703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68780 00000100:00100000:3.0:1787826429.764932:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826429.765021:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332725760 00000100:00100000:2.0:1787826429.765033:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332725760 00000100:00100000:2.0:1787826429.765133:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68781 00000100:00100000:2.0:1787826429.765146:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+9:229971:x1874663332725760:12345-192.168.203.56@tcp:36:touch.0 00010000:00010000:2.0:1787826429.765223:0:24702:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787826429.765255:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43000/0xe21309aa81f4cd85 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dc5 expref: 9 pid: 24703 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826429.765276:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f794b57a600) 00010000:00010000:2.0:1787826429.765292:0:24702:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787826429.765366:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4cd93 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826429.765382:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4cd93 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826429.765393:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4cd93 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826429.765524:0:24702:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f309e1b0 reply state 00000000e8bd9446 transno 8589957531 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4cd93 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826429.765537:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef800/0xe21309aa81f4cd93 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826429.765543:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826429.765546:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826429.765549:0:24702:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787826429.765664:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851240000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:229971:x1874663332725760:12345-192.168.203.56@tcp:36:touch.0 Request processed in 520us (746us total) trans 8589957531 rc 0/0 00000100:00100000:2.0:1787826429.765672:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68781 00000100:00100000:1.0:1787826429.771100:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826429.771213:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332725888 00000100:00100000:2.0:1787826429.771225:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332725888 00000100:00100000:2.0:1787826429.771296:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68782 00000100:00100000:2.0:1787826429.771309:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+10:229971:x1874663332725888:12345-192.168.203.56@tcp:101:touch.0 00010000:00010000:2.0:1787826429.771339:0:17813:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826429.771366:0:17813:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4cd9a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801dcc expref: -99 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826429.771435:0:17813:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4cd9a lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6eb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dcc expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826429.771451:0:17813:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4cd9a lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x6eb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dcc expref: 11 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826429.771470:0:17813:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826429.771474:0:17813:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787826429.771478:0:17813:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490ef400, rc 0) 00000100:00100000:2.0:1787826429.771539:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784b750000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:229971:x1874663332725888:12345-192.168.203.56@tcp:101:touch.0 Request processed in 233us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826429.771546:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68782 00000100:00100000:0.0:1787826429.789133:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826429.789253:0:8759:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332726016 00000100:00100000:1.0:1787826429.789264:0:8759:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332726016 00000100:00080000:0.0:1787826429.791120:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826429.791306:0:8759:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30918 00000100:00100000:1.0:1787826429.791330:0:8759:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2395:x1874663332726016:12345-192.168.203.56@tcp:2:touch.0 00000100:00100000:1.0:1787826429.791609:0:8759:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2395:x1874663332726016:12345-192.168.203.56@tcp:2:touch.0 Request processed in 289us (2508us total) trans 4294976178 rc 0/0 00000100:00100000:1.0:1787826429.791633:0:8759:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30918 00000100:00100000:2.0:1787826429.808393:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826429.811993:0:6530:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332726272 00000100:00100000:0.0:1787826429.812010:0:6530:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332726272 00000100:00100000:2.0:1787826429.812327:0:6530:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9414 00000100:00100000:2.0:1787826429.812342:0:6530:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+11:229971:x1874663332726272:12345-192.168.203.56@tcp:35:touch.0 00000100:00100000:2.0:1787826429.812700:0:6530:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:229971:x1874663332726272:12345-192.168.203.56@tcp:35:touch.0 Request processed in 359us (4318us total) trans 8589957532 rc 0/0 00000100:00100000:2.0:1787826429.812709:0:6530:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9414 00010000:00010000:3.0:1787826430.124365:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826431.149647:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826431.230421:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826431.232325:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332726400 00000100:00100000:1.0:1787826431.232348:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332726400 00000100:00100000:1.0:1787826431.232641:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68783 00000100:00100000:1.0:1787826431.232656:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+12:230034:x1874663332726400:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:1.0:1787826431.232684:0:6527:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826431.232706:0:6527:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cda1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801dd3 expref: -99 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.232779:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cda8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.232793:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cda8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.232805:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cda8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787826431.232858:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:1.0:1787826431.232862:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787826431.232967:0:6527:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cda8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.232994:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecf800/0xe21309aa81f4cda1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dd3 expref: 14 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.233012:0:6527:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cda8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dd3 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.233021:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:1.0:1787826431.233024:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787826431.233028:0:6527:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ece600, rc 0) 00000100:00100000:1.0:1787826431.233183:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:230034:x1874663332726400:12345-192.168.203.56@tcp:101:link.0 Request processed in 530us (2771us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826431.233190:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68783 00000100:00080000:3.0:1787826431.236565:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787826431.252235:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826431.252331:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332726528 00000100:00100000:0.0:1787826431.252350:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332726528 00000100:00100000:0.0:1787826431.252420:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68784 00000100:00100000:0.0:1787826431.252432:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:230034:x1874663332726528:12345-192.168.203.56@tcp:101:link.0 00010000:00010000:0.0:1787826431.252463:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826431.252486:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cdaf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801dda expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252562:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252577:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252588:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252599:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252606:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252613:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252706:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4cdc4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252716:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4cdc4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252723:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4cdc4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826431.252763:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6eb:0x0]: returning size 0 00000004:00200000:0.0:1787826431.252768:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826431.252789:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4cdc4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252804:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4cdc4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252812:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252829:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252834:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252840:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252846:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43c00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826431.252851:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252857:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252878:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826431.252882:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826431.252886:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252895:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43c00/0xe21309aa81f4cdb6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252909:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252915:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252920:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252925:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43a00 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826431.252927:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252932:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252945:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x14efb6f5 (ffff9f794b57a000) 00010000:00010000:0.0:1787826431.252948:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826431.252950:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252956:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43a00/0xe21309aa81f4cdbd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252982:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42600/0xe21309aa81f4cdaf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dda expref: 15 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.252995:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4cdc4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dda expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826431.253002:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:0.0:1787826431.253004:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826431.253007:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42400, rc 0) 00000100:00100000:0.0:1787826431.253074:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:230034:x1874663332726528:12345-192.168.203.56@tcp:101:link.0 Request processed in 644us (849us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826431.253081:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68784 00000100:00100000:2.0:1787826431.262649:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826431.263193:0:35080:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332726784 00000100:00100000:3.0:1787826431.263204:0:35080:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332726784 00000100:00100000:3.0:1787826431.263277:0:35080:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68785 00000100:00100000:3.0:1787826431.263288:0:35080:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:230034:x1874663332726784:12345-192.168.203.56@tcp:36:link.0 00010000:00010000:3.0:1787826431.263525:0:35080:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1787826431.263557:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42400/0xe21309aa81f4cdc4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dda expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826431.263585:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:3.0:1787826431.263598:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ece600/0xe21309aa81f4cda8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dd3 expref: 13 pid: 6527 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826431.263609:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826431.263612:0:35080:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826431.263641:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826431.263656:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826431.263666:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826431.263682:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cdd2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826431.263689:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cdd2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826431.263695:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cdd2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00020000:3.0:1787826431.263702:0:35080:0:(mdt_handler.c:4171:mdt_object_pdo_lock()) cfs_fail_timeout id 145 sleeping for 15000ms 00000100:00100000:3.0:1787826431.747406:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826431.748068:0:35095:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332726912 00000100:00100000:1.0:1787826431.748081:0:35095:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332726912 00000100:00100000:1.0:1787826431.748198:0:35095:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68786 00000100:00080000:3.0:1787826431.748214:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826431.748221:0:35095:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:230036:x1874663332726912:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:1.0:1787826431.748253:0:35095:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787826431.748277:0:35095:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cdd9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801de1 expref: -99 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.748348:0:35095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.748360:0:35095:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 2/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.748369:0:35095:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.748388:0:35095:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826431.748400:0:35095:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826432.172953:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826432.221643:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826432.221700:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826432.221733:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826432.221766:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826432.221784:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826432.221831:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826432.221852:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332727552 00000100:00100000:0.0:1787826432.221865:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:3.0:1787826432.221876:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332727552 00000100:00100000:0.0:1787826432.221904:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332727296 00000100:00100000:0.0:1787826432.221915:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332727296 00000100:00100000:0.0:1787826432.221984:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332727808 00000100:00100000:0.0:1787826432.221987:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332727808 00000100:00100000:0.0:1787826432.222008:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332727168 00000100:00100000:0.0:1787826432.222011:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332727168 00000100:00100000:0.0:1787826432.222024:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332727680 00000100:00100000:0.0:1787826432.222027:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332727680 00000100:00100000:0.0:1787826432.222047:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14507 00000100:00100000:0.0:1787826432.222056:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332727296:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826432.222194:0:35096:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332727424 00000100:00100000:3.0:1787826432.222201:0:35096:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332727424 00000100:00100000:0.0:1787826432.222225:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2394:x1874663332727296:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 166us (521us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826432.222238:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14507 00000100:00100000:3.0:1787826432.222249:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8964 00000100:00100000:0.0:1787826432.222271:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14508 00000100:00100000:3.0:1787826432.222272:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332727552:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826432.222290:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332727808:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826432.222338:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+5:2394:x1874663332727808:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 54us (569us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826432.222345:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14508 00000100:00100000:0.0:1787826432.222356:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14509 00000100:00100000:0.0:1787826432.222362:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:2393:x1874663332727168:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:3.0:1787826432.222446:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784836ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2394:x1874663332727552:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 183us (708us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826432.222453:0:30899:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332727040 00000100:00100000:3.0:1787826432.222461:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8964 00000100:00100000:2.0:1787826432.222463:0:30899:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332727040 00000100:00100000:3.0:1787826432.222498:0:35096:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 8965 00000100:00100000:3.0:1787826432.222511:0:35096:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332727424:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:2.0:1787826432.222525:0:30899:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 2268 00000100:00100000:2.0:1787826432.222536:0:30899:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847250850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332727040:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:1.0:1787826432.222636:0:35093:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 14510 00000100:00100000:0.0:1787826432.222658:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:398e0287-d3c7-44dd-b1d9-6b85f698e615+13:2393:x1874663332727168:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 293us (870us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826432.222664:0:35093:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332727680:12345-192.168.203.56@tcp:400:kworker.0 00000100:00100000:0.0:1787826432.222670:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14509 00000100:00100000:3.0:1787826432.222719:0:35096:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:398e0287-d3c7-44dd-b1d9-6b85f698e615+5:2393:x1874663332727424:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 207us (884us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826432.222726:0:35093:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_010:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:2393:x1874663332727680:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 75us (857us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826432.222736:0:35096:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 8965 00000100:00080000:3.0:1787826432.222776:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787826432.222783:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1787826432.222786:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826432.223339:0:35093:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 14510 00000100:00100000:2.0:1787826432.228328:0:30899:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847250850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:fd94cc3d-2daf-4fbe-98ba-f53926e75993+13:2394:x1874663332727040:12345-192.168.203.56@tcp:400:kworker.0 Request processed in 5781us (6684us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826432.228345:0:30899:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 2268 00000100:00100000:1.0:1787826433.004466:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346394368:0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.004553:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826433.004601:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346394880:0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.004638:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826433.004667:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346395392:0@lo:400:kworker.0 00000100:00100000:2.0:1787826433.004668:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784b750e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346394240:0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.004694:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826433.004720:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826433.004731:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1787826433.004739:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346394496:0@lo:400:kworker.0 00000100:00100000:2.0:1787826433.004762:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346394752:0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.004771:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826433.004797:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346395008:0@lo:400:kworker.0 00000100:00100000:2.0:1787826433.004801:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826433.004842:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7845afc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346394624:0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.004843:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826433.004844:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346395264:0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.004874:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346395520:0@lo:400:kworker.0 00000100:00100000:2.0:1787826433.004887:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826433.004900:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826433.004911:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826433.004916:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346395776:0@lo:400:kworker.0 00000100:00100000:3.0:1787826433.004938:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9eb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346395136:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.005096:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346394880 00000100:00100000:0.0:1787826433.005175:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346394880 00000100:00100000:0.0:1787826433.005239:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346395264 00000100:00100000:0.0:1787826433.005242:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346395264 00000100:00100000:0.0:1787826433.005261:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346395520 00000100:00100000:0.0:1787826433.005264:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346395520 00000100:00100000:0.0:1787826433.005287:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8966 00000100:00100000:0.0:1787826433.005297:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346394880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.005342:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346394368 00000100:00100000:1.0:1787826433.005347:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346394368 00000100:00100000:0.0:1787826433.005381:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3652:x1874663346394880:12345-0@lo:400:kworker.0 Request processed in 85us (740us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826433.005381:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346395392 00000100:00100000:1.0:1787826433.005385:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346395392 00000100:00100000:0.0:1787826433.005394:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8966 00000100:00100000:1.0:1787826433.005412:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24283 00000100:00100000:0.0:1787826433.005420:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8967 00000100:00100000:1.0:1787826433.005424:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346394368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.005445:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346395264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.005466:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784d9ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346394368:12345-0@lo:400:kworker.0 Request processed in 46us (912us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826433.005473:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24283 00000100:00100000:1.0:1787826433.005490:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24284 00000100:00100000:0.0:1787826433.005508:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346395264:12345-0@lo:400:kworker.0 Request processed in 79us (622us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.005516:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8967 00000100:00100000:0.0:1787826433.005529:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8968 00000100:00100000:0.0:1787826433.005535:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346395520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787826433.005567:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346395392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.005568:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346395520:12345-0@lo:400:kworker.0 Request processed in 32us (657us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.005580:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8968 00000100:00100000:0.0:1787826433.005605:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346394496 00000100:00100000:0.0:1787826433.005609:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346394496 00000100:00100000:0.0:1787826433.005639:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346394752 00000100:00100000:0.0:1787826433.005642:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346394752 00000100:00100000:1.0:1787826433.005661:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1874663346395392:12345-0@lo:400:kworker.0 Request processed in 53us (919us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.005666:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346395008 00000100:00100000:0.0:1787826433.005671:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346395008 00000100:00100000:1.0:1787826433.005672:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24284 00000100:00100000:0.0:1787826433.005697:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346394624 00000100:00100000:0.0:1787826433.005701:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346394624 00000100:00100000:0.0:1787826433.005726:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14511 00000100:00100000:0.0:1787826433.005733:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346394496:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.005776:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874663346394496:12345-0@lo:400:kworker.0 Request processed in 39us (1001us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.005783:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14511 00000100:00100000:0.0:1787826433.005794:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14512 00000100:00100000:0.0:1787826433.005800:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346394752:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.005841:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3655:x1874663346394752:12345-0@lo:400:kworker.0 Request processed in 42us (1045us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.005847:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14512 00000100:00100000:0.0:1787826433.005857:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14513 00000100:00100000:0.0:1787826433.005863:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346395008:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.005886:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3652:x1874663346395008:12345-0@lo:400:kworker.0 Request processed in 24us (1048us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.005892:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14513 00000100:00100000:0.0:1787826433.005901:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14514 00000100:00100000:0.0:1787826433.005907:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346394624:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.005933:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3654:x1874663346394624:12345-0@lo:400:kworker.0 Request processed in 27us (1036us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.005939:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14514 00000100:00080000:0.0:1787826433.005960:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787826433.005965:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787826433.005968:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787826433.005970:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787826433.006037:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874663346394496:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.006050:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3652:1874663346395008:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.006059:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294973870 gen 2 00000004:00080000:0.0:1787826433.006064:0:3652:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9f7851243100, transno 4294973870 00000100:00100000:0.0:1787826433.006076:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346395520:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.006086:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b752d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346394368:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.006095:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346394880:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.006127:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3652:1874663346395392:0@lo:400:kworker.0 00000100:00100000:2.0:1787826433.007400:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826433.007446:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346395776 00000100:00100000:0.0:1787826433.007454:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346395776 00000100:00100000:2.0:1787826433.007475:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3655:1874663346394752:0@lo:400:kworker.0 00000100:00100000:2.0:1787826433.007491:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346395264:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.007503:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14515 00000100:00100000:0.0:1787826433.007509:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346395776:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.007551:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3655:x1874663346395776:12345-0@lo:400:kworker.0 Request processed in 42us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.007555:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14515 00000100:00100000:2.0:1787826433.007563:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9ea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3655:1874663346395776:0@lo:400:kworker.0 00000100:00100000:2.0:1787826433.007611:0:30900:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346394240 00000100:00100000:2.0:1787826433.007615:0:30900:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346394240 00000100:00100000:2.0:1787826433.007654:0:30900:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2269 00000100:00100000:2.0:1787826433.007660:0:30900:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847251850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346394240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787826433.007701:0:30900:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847251850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4305192b-43a6-4bf4-9f25-68faf720ed51+24:3655:x1874663346394240:12345-0@lo:400:kworker.0 Request processed in 42us (2984us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826433.007706:0:30900:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2269 00000100:00100000:1.0:1787826433.007754:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784b750e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4305192b-43a6-4bf4-9f25-68faf720ed51:3655:1874663346394240:0@lo:400:kworker.0 00000100:00100000:3.0:1787826433.007956:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826433.008028:0:3654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784d9e8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346395648:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.008075:0:17813:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346395136 00000100:00100000:0.0:1787826433.008080:0:17813:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346395136 00000100:00100000:0.0:1787826433.008242:0:17813:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14516 00000100:00100000:0.0:1787826433.008249:0:17813:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346395136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.008284:0:17813:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3654:x1874663346395136:12345-0@lo:400:kworker.0 Request processed in 36us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.008290:0:17813:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14516 00000100:00080000:2.0:1787826433.008345:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787826433.008448:0:3654:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826433.008490:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7845afc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3654:1874663346394624:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.008537:0:35097:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346395648 00000100:00100000:0.0:1787826433.008542:0:35097:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346395648 00000100:00100000:0.0:1787826433.008590:0:35097:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8969 00000100:00100000:0.0:1787826433.008597:0:35097:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346395648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787826433.008617:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9eb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3654:1874663346395136:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.008629:0:35097:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0000-mdtlov_UUID+5:3654:x1874663346395648:12345-0@lo:400:kworker.0 Request processed in 33us (181us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826433.008629:0:3654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784d9e8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3654:1874663346395648:0@lo:400:kworker.0 00000100:00100000:0.0:1787826433.008635:0:35097:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8969 00000100:00080000:2.0:1787826433.008685:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787826433.197346:0:30445:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00020000:2.0:1787826433.261360:0:35080:0:(mdt_handler.c:4171:mdt_object_pdo_lock()) cfs_fail_timeout interrupted 00010000:00010000:2.0:1787826433.261409:0:35080:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4cde7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.261429:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4cde7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.261447:0:35080:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4cde7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826433.261823:0:35080:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007f321217 reply state 00000000ff12d537 transno 8589957533 ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4cde7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262282:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee400/0xe21309aa81f4cde7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x6eb:0x0].0x0 bits 0x22/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262296:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826433.262300:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00000100:00100000:2.0:1787826433.262304:0:35080:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787826433.262313:0:35080:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262320:0:35080:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262326:0:35080:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262333:0:35080:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262338:0:35080:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826433.262343:0:35080:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262350:0:35080:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262362:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826433.262366:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826433.262370:0:35080:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262378:0:35080:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262385:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262424:0:35080:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262431:0:35080:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69200/0xe21309aa81f4cdcb lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787826433.262444:0:35080:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007f321217 reply state 00000000ff12d537 transno 8589957533 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cdd2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262453:0:35080:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69600/0xe21309aa81f4cdd2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 35080 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.262459:0:35080:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xab5f4fa8 (ffff9f79460a2500) 00010000:00010000:2.0:1787826433.262462:0:35080:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787826433.262464:0:35080:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00010000:00010000:3.0:1787826433.262521:0:35095:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.262531:0:35095:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826433.262558:0:35080:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_008:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:230034:x1874663332726784:12345-192.168.203.56@tcp:36:link.0 Request processed in 1999267us (1999916us total) trans 8589957533 rc 0/0 00000100:00100000:2.0:1787826433.262568:0:35080:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68785 00000004:00200000:3.0:1787826433.262570:0:35095:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:3.0:1787826433.262576:0:35095:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787826433.262595:0:35095:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.262625:0:35095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecee00/0xe21309aa81f4cdd9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801de1 expref: 8 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.262643:0:35095:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801de1 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.262651:0:35095:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826433.262654:0:35095:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787826433.262657:0:35095:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7848ecec00, rc 0) 00000100:00100000:3.0:1787826433.262685:0:35095:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977c6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_012:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:230036:x1874663332726912:12345-192.168.203.56@tcp:101:rm.0 Request processed in 1514470us (1515290us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826433.262694:0:35095:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68786 00000100:00100000:2.0:1787826433.269469:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826433.274332:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332727936 00000100:00100000:0.0:1787826433.274347:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332727936 00000100:00100000:0.0:1787826433.274695:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68787 00000100:00100000:0.0:1787826433.274706:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:230036:x1874663332727936:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826433.275295:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826433.275319:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4cdee lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801de8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826433.275889:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1787826433.276595:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276613:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276625:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276642:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276649:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276655:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276753:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce03 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276762:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276768:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826433.276824:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6eb:0x0]: returning size 0 00000004:00200000:0.0:1787826433.276829:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826433.276850:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276862:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276872:0:6528:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276878:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276888:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276894:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276900:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43200 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826433.276905:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276912:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276925:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826433.276928:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826433.276932:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276938:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4cdf5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276949:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276955:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276961:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276966:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826433.276969:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276975:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276982:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xab5f4fa8 (ffff9f79460a2500) 00010000:00010000:0.0:1787826433.276984:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826433.276987:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.276993:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4cdfc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.277026:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4cdee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801de8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.277039:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce03 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801de8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.277047:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:0.0:1787826433.277050:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787826433.277052:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826433.277142:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7847b5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:230036:x1874663332727936:12345-192.168.203.56@tcp:101:rm.0 Request processed in 2436us (7680us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.277151:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68787 00000100:00100000:0.0:1787826433.281427:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:1.0:1787826433.281578:0:35352:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332728064 00000100:00100000:1.0:1787826433.281589:0:35352:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332728064 00000100:00100000:1.0:1787826433.281648:0:35352:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30919 00000100:00100000:1.0:1787826433.281657:0:35352:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:0+-99:2392:x1874663332728064:12345-192.168.203.56@tcp:8:rm.0 00010000:00080000:1.0:1787826433.281693:0:35352:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from fc0b407b-a541-420d-8712-b9c8da18fe8c@192.168.203.56@tcp t0 exp 0000000000000000 cur 8245 last 0 00000020:00000080:1.0:1787826433.281738:0:35352:0:(genops.c:1385:class_connect()) connect: client fc0b407b-a541-420d-8712-b9c8da18fe8c, cookie 0xe21309aa81f4ce0a 00002000:00100000:1.0:1787826433.281750:0:35352:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli fc0b407b-a541-420d-8712-b9c8da18fe8c/ffff9f7844d35000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113950 ocd_grant: 484638720 ocd_index: 0 ocd_group 0 00002000:00100000:1.0:1787826433.281767:0:35352:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:1.0:1787826433.281839:0:35352:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:1.0:1787826433.281872:0:35352:0:(import.c:65:import_set_state_nolock()) ffff9f7844d34800 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1787826433.281939:0:35352:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:2392:x1874663332728064:12345-192.168.203.56@tcp:8:rm.0 Request processed in 282us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826433.281947:0:35352:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30919 00000100:00100000:1.0:1787826433.286402:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826433.286494:0:20167:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332728192 00000100:00100000:0.0:1787826433.286504:0:20167:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332728192 00000100:00100000:0.0:1787826433.286582:0:20167:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30920 00000100:00100000:0.0:1787826433.286594:0:20167:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+6:0:x1874663332728192:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826433.286611:0:20167:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826433.286644:0:20167:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4ce11 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x526:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801def expref: -99 pid: 20167 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787826433.286671:0:20167:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x526:0x0] initial LVB size: 0, mtime: 0x6a9010fc, atime: 0x6a9010fc, ctime: 0x6a9010fc, blocks: 0x0 00010000:00010000:0.0:1787826433.286689:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x526:0x0].0x0 since=8245s v=(0 0) 00010000:00010000:0.0:1787826433.286694:0:20167:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x526:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826433.286703:0:20167:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4ce11 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x526:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801def expref: 7 pid: 20167 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.286721:0:20167:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f0d22958/0xe21309aa81f4ce11 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x526:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801def expref: 7 pid: 20167 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.286734:0:20167:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43e00, rc 0) 00000100:00100000:0.0:1787826433.286784:0:20167:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:fc0b407b-a541-420d-8712-b9c8da18fe8c+7:0:x1874663332728192:12345-192.168.203.56@tcp:101:rm.0 Request processed in 193us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.286791:0:20167:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30920 00000100:00080000:1.0:1787826433.286919:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787826433.289999:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826433.290029:0:24702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332728448 00000100:00100000:0.0:1787826433.290039:0:24702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332728448 00000100:00100000:0.0:1787826433.290100:0:24702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68788 00000100:00100000:0.0:1787826433.290131:0:24702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:230036:x1874663332728448:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826433.290185:0:24702:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1787826433.290215:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce03 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801de8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.290237:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:0.0:1787826433.290250:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7848ecec00/0xe21309aa81f4cde0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801de1 expref: 7 pid: 35095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.290259:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826433.290262:0:24702:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787826433.290345:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290361:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290388:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290404:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4ce1f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290421:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4ce1f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290429:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4ce1f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290561:0:24702:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290571:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290578:0:24702:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290875:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290883:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290889:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290896:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e69000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826433.290900:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290912:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290922:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:3.0:1787826433.290924:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787826433.290928:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290934:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69000/0xe21309aa81f4ce26 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290950:0:24702:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290956:0:24702:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290961:0:24702:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290967:0:24702:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290972:0:24702:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7846e68600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:3.0:1787826433.290975:0:24702:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.290981:0:24702:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.291001:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:3.0:1787826433.291005:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787826433.291007:0:24702:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.291013:0:24702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e68600/0xe21309aa81f4ce18 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787826433.291022:0:24702:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000024ecf2ed reply state 00000000f13d7417 transno 8589957534 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4ce1f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.291033:0:24702:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7846e69e00/0xe21309aa81f4ce1f lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xab5f4fa8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 24702 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.291039:0:24702:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xab5f4fa8 (ffff9f79460a2500) 00010000:00010000:3.0:1787826433.291042:0:24702:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787826433.291045:0:24702:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787826433.291902:0:24702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851886680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:fc0b407b-a541-420d-8712-b9c8da18fe8c+8:230036:x1874663332728448:12345-192.168.203.56@tcp:36:rm.0 Request processed in 1767us (1908us total) trans 8589957534 rc 0/0 00000100:00100000:3.0:1787826433.291914:0:24702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68788 00000100:00100000:2.0:1787826433.516616:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826433.516709:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346395904:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787826433.516782:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787826433.516919:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346395904 00000100:00100000:3.0:1787826433.516928:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346395904 00000100:00100000:3.0:1787826433.516984:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6319 00000100:00100000:3.0:1787826433.516994:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346395904:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1787826433.517024:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 17082944 pending 0 free 3914715136 avail 3683192832 00000100:00100000:3.0:1787826433.517075:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874663346395904:12345-0@lo:13:osp-pre-1-1.0 Request processed in 83us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826433.517082:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6319 00000100:00100000:2.0:1787826433.517180:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3653:1874663346395904:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787826433.836442:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826433.836588:0:35098:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332728576 00000100:00100000:0.0:1787826433.836599:0:35098:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332728576 00000100:00100000:0.0:1787826433.836676:0:35098:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68789 00000100:00100000:0.0:1787826433.836687:0:35098:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+14:229720:x1874663332728576:12345-192.168.203.56@tcp:101:bash.0 00010000:00010000:0.0:1787826433.836713:0:35098:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826433.836739:0:35098:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce2d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801df6 expref: -99 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.836817:0:35098:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ce34 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.836830:0:35098:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ce34 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.836843:0:35098:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ce34 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787826433.836884:0:35098:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 73728 00000004:00200000:0.0:1787826433.836887:0:35098:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787826433.836905:0:35098:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ce34 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 35098 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.836932:0:35098:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42e00/0xe21309aa81f4ce2d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801df6 expref: 16 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.836947:0:35098:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ce34 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801df6 expref: 15 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.837186:0:35098:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826433.837189:0:35098:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787826433.837192:0:35098:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d43800, rc 0) 00000100:00100000:0.0:1787826433.837253:0:35098:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851884a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_013:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:229720:x1874663332728576:12345-192.168.203.56@tcp:101:bash.0 Request processed in 567us (819us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.837260:0:35098:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68789 00000100:00080000:1.0:1787826433.837700:0:6523:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787826433.841161:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826433.841292:0:17601:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332728704 00000100:00100000:2.0:1787826433.841302:0:17601:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332728704 00000100:00100000:2.0:1787826433.841367:0:17601:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 9415 00000100:00100000:2.0:1787826433.841377:0:17601:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:229720:x1874663332728704:12345-192.168.203.56@tcp:37:bash.0 00000100:00100000:2.0:1787826433.856888:0:17601:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7851887100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:229720:x1874663332728704:12345-192.168.203.56@tcp:37:bash.0 Request processed in 15502us (15778us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826433.856902:0:17601:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 9415 00000100:00100000:1.0:1787826433.872897:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:2.0:1787826433.873207:0:35094:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332728832 00000100:00100000:2.0:1787826433.873218:0:35094:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332728832 00000100:00100000:2.0:1787826433.873345:0:35094:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68790 00000100:00100000:2.0:1787826433.873357:0:35094:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:230078:x1874663332728832:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:2.0:1787826433.873378:0:35094:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787826433.873400:0:35094:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ce3b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801dfd expref: -99 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873594:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873607:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873618:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873630:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873652:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873661:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873756:0:35094:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ce50 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873765:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ce50 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873772:0:35094:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ce50 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787826433.873828:0:35094:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x6eb:0x0]: returning size 0 00000004:00200000:2.0:1787826433.873834:0:35094:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787826433.873855:0:35094:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ce50 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873867:0:35094:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ce50 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873877:0:35094:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873883:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873890:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873897:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873907:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ef000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826433.873912:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873919:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873930:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:2.0:1787826433.873933:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826433.873938:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873944:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef000/0xe21309aa81f4ce42 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873958:0:35094:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873964:0:35094:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873970:0:35094:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873975:0:35094:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f78490ee800 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:2.0:1787826433.873977:0:35094:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873983:0:35094:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.873993:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x14efb6f5 (ffff9f794b57a000) 00010000:00010000:2.0:1787826433.873995:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787826433.873998:0:35094:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.874004:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee800/0xe21309aa81f4ce49 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 35094 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.874036:0:35094:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ee000/0xe21309aa81f4ce3b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dfd expref: 17 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.874049:0:35094:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ce50 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dfd expref: 16 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826433.874056:0:35094:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:2.0:1787826433.874059:0:35094:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787826433.874061:0:35094:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f78490eee00, rc 0) 00000100:00100000:2.0:1787826433.874175:0:35094:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_011:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:230078:x1874663332728832:12345-192.168.203.56@tcp:101:rm.0 Request processed in 821us (1287us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826433.874183:0:35094:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68790 00000100:00100000:2.0:1787826433.879635:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826433.879740:0:8445:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332728960 00000100:00100000:0.0:1787826433.879757:0:8445:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332728960 00000100:00100000:0.0:1787826433.879824:0:8445:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 30921 00000100:00100000:0.0:1787826433.879836:0:8445:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+6:2393:x1874663332728960:12345-192.168.203.56@tcp:101:rm.0 00010000:00010000:0.0:1787826433.879917:0:8445:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787826433.879938:0:8445:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000329fecdf/0xe21309aa81f4ce57 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x526:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x95194e8760801e04 expref: -99 pid: 8445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.879973:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x526:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1787826433.879978:0:8445:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x526:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787826433.879986:0:8445:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000329fecdf/0xe21309aa81f4ce57 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x526:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801e04 expref: 7 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.880005:0:8445:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000329fecdf/0xe21309aa81f4ce57 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x526:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801e04 expref: 7 pid: 8445 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.880019:0:8445:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9f7845d42e00, rc 0) 00000100:00100000:0.0:1787826433.880077:0:8445:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:398e0287-d3c7-44dd-b1d9-6b85f698e615+7:2393:x1874663332728960:12345-192.168.203.56@tcp:101:rm.0 Request processed in 244us (451us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826433.880084:0:8445:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 30921 00000100:00100000:1.0:1787826433.883223:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.56@tcp (source: 12345-192.168.203.56@tcp) 00000100:00100000:0.0:1787826433.883941:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663332729216 00000100:00100000:0.0:1787826433.883952:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663332729216 00000100:00100000:0.0:1787826433.884092:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.56@tcp, seq: 68791 00000100:00100000:0.0:1787826433.884464:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+16:230078:x1874663332729216:12345-192.168.203.56@tcp:36:rm.0 00010000:00010000:0.0:1787826433.884522:0:6527:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1787826433.884552:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490eee00/0xe21309aa81f4ce50 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6eb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dfd expref: 16 pid: 35094 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.884575:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f78490ef400/0xe21309aa81f4cd9a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6eb:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801dcc expref: 15 pid: 17813 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.884747:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:0.0:1787826433.884763:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43800/0xe21309aa81f4ce34 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.56@tcp remote: 0x95194e8760801df6 expref: 14 pid: 35098 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.884775:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826433.884778:0:6527:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787826433.884808:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.884821:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.884832:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.884844:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ce65 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.884926:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ce65 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.884937:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ce65 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.885093:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.885102:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.885150:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787826433.885970:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0xa:0x0].219, 64 off22208 00000004:00080000:0.0:1787826433.885982:0:6527:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 8246097876273ns, next at 8247097873258ns, rc = 0 00010000:00010000:0.0:1787826433.886038:0:6527:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886047:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010080000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886055:0:6527:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40010080000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886066:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40010080000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886072:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010480000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886078:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x50010480000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886101:0:6527:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787826433.886489:0:7478:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826433.886496:0:7478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40010480000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.886509:0:7478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42a00 already unused, calling callback (ffffffffc0d4d230) 00010000:00010000:3.0:1787826433.886513:0:7478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40010480000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.886520:0:7478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 5/0,0 mode: PW/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x40018480000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.886531:0:7478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:3.0:1787826433.886534:0:7478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1787826433.886538:0:7478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 3/0,0 mode: --/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x44819480000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826433.886546:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826433.886548:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826433.886549:0:7478:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826433.886748:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42a00/0xe21309aa81f4ce73 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x6eb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT flags: 0x44819480000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886780:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886786:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886791:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886806:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d43600 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826433.886810:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886816:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886825:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6eb:0x0].0x0 (ffff9f784abfee00) 00010000:00010000:0.0:1787826433.886828:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787826433.886831:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886837:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43600/0xe21309aa81f4ce6c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6eb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886912:0:6527:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886920:0:6527:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886925:0:6527:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886930:0:6527:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886934:0:6527:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9f7845d42000 already unused, calling callback (ffffffffc197a4f0) 00010000:00010000:0.0:1787826433.886938:0:6527:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.886943:0:6527:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.887051:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9f7967ac1500) 00010000:00010000:0.0:1787826433.887054:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787826433.887058:0:6527:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.887063:0:6527:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d42000/0xe21309aa81f4ce5e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787826433.887074:0:6527:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004c6cbc9f reply state 0000000058c759d2 transno 8589957535 ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ce65 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.887087:0:6527:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9f7845d43200/0xe21309aa81f4ce65 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x14efb6f5 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6527 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826433.887093:0:6527:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x14efb6f5 (ffff9f794b57a000) 00010000:00010000:0.0:1787826433.887095:0:6527:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787826433.887099:0:6527:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787826433.887223:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daa9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:398e0287-d3c7-44dd-b1d9-6b85f698e615+15:230078:x1874663332729216:12345-192.168.203.56@tcp:36:rm.0 Request processed in 3109us (3998us total) trans 8589957535 rc 0/0 00000100:00100000:0.0:1787826433.887232:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.56@tcp, seq: 68791 00000100:00100000:1.0:1787826434.028400:0:3655:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787826434.028441:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346396288:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826434.028450:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977c6d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346396032:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826434.028560:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7850ad7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346396416:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787826434.029285:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787826434.029375:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826434.029393:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f7977dca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346396544:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787826434.029410:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826434.029428:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787826434.029453:0:3655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9f784daab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346396160:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787826434.029478:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346396288 00000100:00100000:1.0:1787826434.029484:0:3655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787826434.029486:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346396288 00000100:00100000:1.0:1787826434.029522:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346396544 00000100:00100000:1.0:1787826434.029527:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346396544 00000100:00100000:2.0:1787826434.029530:0:10390:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346396160 00000100:00100000:2.0:1787826434.029536:0:10390:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346396160 00000100:00100000:1.0:1787826434.029558:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6320 00000100:00100000:2.0:1787826434.029571:0:10390:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6321 00000100:00100000:1.0:1787826434.029572:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346396288:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787826434.029584:0:10390:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346396544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787826434.029600:0:10390:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 17082944 pending 0 free 3914715136 avail 3683192832 00002000:00000024:1.0:1787826434.029605:0:8447:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:2.0:1787826434.029657:0:10390:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346396544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826434.029660:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3655:x1874663346396288:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (250us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826434.029669:0:10390:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6321 00000100:00100000:3.0:1787826434.029855:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346396032 00000100:00100000:3.0:1787826434.029860:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346396032 00000100:00100000:3.0:1787826434.029891:0:6532:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663346396416 00000100:00100000:3.0:1787826434.029894:0:6532:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663346396416 00000100:00100000:3.0:1787826434.029915:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24285 00000100:00100000:3.0:1787826434.029922:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346396032:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787826434.029947:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1443373056 avail 1297133568 00000100:00100000:3.0:1787826434.030170:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7850ad6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874663346396032:12345-0@lo:41:osp-pre-1-0.0 Request processed in 248us (885us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826434.030178:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24285 00000100:00100000:3.0:1787826434.030196:0:6532:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24286 00000100:00100000:3.0:1787826434.030204:0:6532:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346396416:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787826434.030215:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6320 00000004:00000024:3.0:1787826434.030223:0:6532:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4194304 pending 0 free 1442746368 avail 1296506880 00000100:00100000:3.0:1787826434.030255:0:6532:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f7977dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874663346396416:12345-0@lo:41:osp-pre-0-1.0 Request processed in 53us (881us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826434.030261:0:6532:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24286 00000100:00100000:3.0:1787826434.030317:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977dca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346396544:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787826434.030334:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7977c6d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3653:1874663346396032:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787826434.030367:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3652:1874663346396416:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787826434.030388:0:8446:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6322 00000100:00100000:3.0:1787826434.030398:0:8446:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346396160:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787826434.030410:0:8446:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8559168 pending 0 free 3893981184 avail 3662458880 00000100:00100000:0.0:1787826434.030427:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f7850ad6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3655:1874663346396288:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787826434.030448:0:8446:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9f784daaa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3655:x1874663346396160:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (962us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826434.030449:0:3655:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9f784daab480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3655:1874663346396160:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787826434.030460:0:8446:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6322 00010000:00010000:3.0:1787826434.221502:0:6514:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826435.244272:0:6515:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 70295 lines, 70295 kept, 0 dropped, 0 bad.